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SQL Mag readers Jameel Ahmed, Kristen 
Cheyney, Shaunt Khaldtiance, Bill Lescher, 
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and Chandra Sekhar Pathivada share their real-world solutions. Learn 
about detecting broken database objects, determining the average number 
of days between customers’ orders, importing and exporting SSIS packages, 
finding strings, and avoiding referential integrity 


errors when deleting database records. 
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Set-Based vs. Cursor-Based 
Solutions for Running 
Aggregates 

—lItzik Ben-Gan 

Although set-based solutions to running aggregations 
have many advantages over cursor-based solutions, see 
how cursor-based solutions can provide better perfor- 
mance in certain cases, such as when dealing with very 
large partitions. 


SQL Server 2008’s T-SQL 
Development and Debugging 
Features 


—Michael Otey 
Michael explains why SQL Server 20083 IntelliSense, 
debugging, and code outlining features take some of the 


hassle out of writing T-SQL scripts and stored procedures. 


Editor’s Tip 


Readers tell us all the time how 
much they love SQL Mag because 
the articles are so darned practi- 
cal. Did you know that we could 
publish an article describing YOUR solution? Yes, you 
too could become a published author in SQL Mag. 
Send your real-life, step-by-step solutions to tough 
SQL Server problems to mkeller@ sqlmag.com. 
—Sheila Molnar, executive editor 


27 Synchronize Metadata Across 


SQL Server Database Copies 
—Fred Schuff and Don Ritchie 

Manually tracking changes when you have several copies of 
a database can be a huge chore. These step-by-step instruc- 
tions let you produce a report that will do it for you. 


Rebuild Indexes Online 
—Brian Smyk 

Avoid blocking and table locking by using this job to 
rebuild indexes online. 
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Quality Coding vs. Quick 


Coding 


Rw I needed to work on a Java coding 
project, which regular readers will know is 
not my favorite thing to do. To plan for the project 
development I searched through the most popular 
Java IDEs, including Eclipse, JBuilder, and Net- 
Beans, for a feature set comparable to the features 
I use in Microsoft Visual Studio and the Microsoft 
.NET Framework. I was disappointed to find that 
not one of the Java IDEs was even in the same ball- 
park as the Microsoft environment. This reinforced 
my opinion that the Microsoft developer products 
are the best in the industry. I like the rich set of 
components in Visual Studio and .NET that take 
care of the heavy lifting required to implement rich 
UI objects and even bind those interface objects to 
the database. Many times all I need to do to bring 
these components together is use a few lines of 
code in the components’ event handling routines. 


Visual Studio Versus Java 

Although it’s true that Im more familiar with 
Visual Studio than with the Java IDEs, I was sur- 
prised by the complexity of the Java IDEs. I found 
them difficult to use; they implemented only a sub- 
set of the features I’m used to. Java developers are 
free to disagree with me, of course, but I need the 
features Visual Studio offers: rich project templates 
for web windows, mobile devices, web services, and 
even setup projects; a super-rich graphical design 
environment with over 60 different components; 
IntelliSense; integrated debugging; and multiple 
platform targeting. No Java IDE comes close to 
Visual Studio’s total package. And one of the Java 
IDEs was buggy to boot. 


What’s the Problem with 
Productivity? 

A word of caution: the fact that the Visual Studio 
features boost productivity is a potential pitfall. This 
has nothing to do with their code quality, which is 
rock solid; it’s about the quick-fix mindset that they 
could foster. Developers don’t typically have this 
problem using the Java tools because they make 
you work at every aspect of the project. It’s not so 
easy to lapse into the quick-fix mindset when you 
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can’t take big parts of the project for granted. The 
Microsoft marketers tell developers that the Visual 
Studio features enable productivity because when 
we don’t need to write as much code we’re freed up 
to focus on business requirements. That’s definitely 
the right message, but overzealous developers can 
misinterpret it to mean that they can spend less 
time coding all aspects of the project. Sometimes 
this might be the case, but the danger is that de- 
velopers could apply this quick coding mentality to 
the more time-consuming details that really can’t be 
automated or quickly churned out. 

In several development projects I’ve seen eager- 
to-please, overly optimistic developers underesti- 
mate the time required to complete critical project 
milestones. They wind up shortchanging them- 
selves. Instead of spending more time focusing on 
the difficult problems, such as correctly implement- 
ing business requirements and robustly handling 
user errors and exceptions, they complete the in- 
terface on schedule but end up not having enough 
time to handle the hard parts of the project. In all 
but the simplest of projects the end results may 
look good, but they might not work the way users 
expect them to. And they won't be robust enough to 
handle unexpected runtime conditions. 


Good Code Versus Great Code 
Don’t get me wrong, I love the productivity fea- 
tures in Visual Studio, and I’m hopelessly spoiled 
by them. But by the same token I resist thinking 
that the rapid development they enable would 
apply to all aspects of my projects. Remem- 
ber that your ultimate goal should be to 
write better quality code, not to just write 
more code faster. Don’t let Visual Studio’s 
fantastic productivity enhancements beguile 
you into reducing your project development esti- 
mates. Improved productivity is good, but robust 
code that truly solves business problems is great. 
If you want to share your experiences about us- 
ing Visual Studio’s productivity features or the 
Java IDEs, drop us line at motey@sqlmag.com or 
letters@sqlmag.com. SOL 
InstantDoc ID 101881 


€ . 


Michael Otey 


(motey @ sqlmag.com) is technical director 
for Windows IT Pro and SQL Server Maga- 
zine and author of Microsoft SQL Server 

2008 New Features (Osborne/McGraw-Hill). 
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At SQL Server 
Magazine, we 
know you 
want articles that give 
you field-tested, practical 
solutions. Reader to Reader 
content puts experts like 
you directly in contact with 
other reader-experts who 
have successfully solved hard 
problems. Last year, we fea- 
tured four Reader to Reader 
articles as November's cover 
story. We're proud to feature 
five more Reader to Reader 
articles this month. We love 
receiving your practical ad- 
vice from the field, so send 
us your favorite solutions to 
the SQL Server challenges 
you've surmounted. 
—Sheila Molnar, 
executive editor, 
SQL Server Magazine 


Bill Lescher 


database architect, Donlen, 
Northbrook, Illinois, 
BLescher@ Donlen.com 
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Find Strings in Database Code, SQL 
Server Agent Jobs, and SSIS Packages 
T-SQL stored procedure is functional and flexible 


fter reading Michael Berry’s article “Find 
A Any String in Any SQL Code in Any 

Database” (August 2008, InstantDoc ID 
99466), I was inspired to try his Find stored proce- 
dure, which lets you find a string of data in any T-SQL 
code stored in any database on the given SQL Server 
instance. To call this stored procedure, you type 


Find 'mystring' 


where mystring is the string you want to search for. 
The stored procedure then searches for that string in 
all the databases on the server and returns a list of the 
objects that contain it. 

After sharing the Find stored procedure with my 
coworkers, we decided to tweak it. In our jobs, we 
often need to search for strings in not only database 
code but also SQL Server Agent jobs and SQL Server 
Integration Services (SSIS) packages, so we adapted 
the stored procedure, which we renamed sp_Find, 


to perform these additional searches. The sp_Find 
stored procedure searches SQL Server Agent jobs 
by looking through the text stored in all job steps 
and searches SSIS packages saved to msdb. (It won't 
search through SSIS packages if they’re saved to a file 
system or through DTS packages.) You can perform 
one type of search (e.g., only SSIS packages), all three 
types of searches (i.e., database code, SQL Server 
Agent jobs, and SSIS packages), or any combination 
thereof (e.g., database code and SQL Server Agent 
jobs). 

We also adapted the stored procedure so that the 
output includes 100 characters of the code surround- 
ing the search string (i.e., the 50 characters preceding 
the string and the 50 characters following the string). 
You can change the number of characters if desired. 
Finally, we adapted the stored procedure so that you 
can search a specific database rather than all the user 
databases on the server. 

To run sp_Find, you follow the syntax 
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sp_Find 'SearchText', DBName, 
PreviewTextSize, SearchDBsFlag, 
SearchJobsFlag, SearchSSISFlag 


where 

* SearchText is the string you want to search for. 

* DBName is the specific database you want to 
search. When you don't include this parameter, all 
the user databases on the server are searched. 

* PreviewTextSize is how many characters of the 
code surrounding the search string you want 
included in the output. When you don’t include 
this parameter, the stored parameter includes 100 
characters surrounding the search string. 

* SearchD BsFlag specifies whether you want to 
search databases (Y) or not (N). 

* SearchJobsFlag specifies whether you want to 
search SQL Server Agent jobs (Y) or not (N). 

* SearchSSIS Flag specifies whether you want to 
search SISS packages (Y) or not (N). 


So, for example, the code 


sp_Find 'track' 


tells the stored procedure to search through all the da- 
tabases, SQL Server Agent jobs, and SSIS packages 
for the string track. The statement 


sp_Find 'track', 'Common', 58 


tells the stored procedure to search through the data- 
base named Common, the SQL Server Agent jobs, and 
the SSIS packages for the string track and include only 
50 characters surrounding the string in the output. 
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eT 


Figure | shows sample results from this statement. 
The code 


sp_Find 'Track', 


"Common', Null, 'Y', 'N', 'Y' 


tells the stored procedure to search through the database 
named Common and the SSIS packages for the string 
track and include 100 characters surrounding the string 
in the output. As this example shows, you need to use 
Null if there’s a parameter you don’t want to set sand- 
wiched between two parameters that you are setting. 

Because our shop is completely on SQL Server 2005, 
we changed the stored procedure’s queries to reflect the 
latest system catalog. Thus, sp_Find won't work on ear- 
lier SQL Server versions. If you prefer to have a search 
tool that works on earlier versions, it wouldn't be dif- 
ficult to remove the SSIS package logic and revert the 
database tables back to the previous catalog version, us- 
ing the code provided in “Find Any String in Any SQL 
Code in Any Database” (August 2008, InstantDoc ID 
99466) as an example. 

You can download the sp_Find.sql file from the 
SQL Server Magazine website by going to www.sql 
mag.com, entering 101896 in the InstantDoc ID text 
box, and clicking the 101896.zip hotlink. You don’t 
need to customize the code at all, provided you're go- 
ing to run it on SQL Server 2008 or SQL Server 2005. 

Note that we named this stored procedure sp_Find 
so that we could store it in the Master system database 
and call it from any database context. Use this name at 
your own risk—Microsoft could come out with a sys- 
tem procedure with the same name in a future release. 

InstantDoc ID 101896 


Figure | 


Sample results from 
the sp_Find stored 


procedure 
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Easily Import and Export 


SSIS Packages 


Storing packages in the msdb database has its 


SQL Server 2005 and later, you can 


advantages 
create a SQL Server Integration Ser- 


l n vices (SSIS) package in Microsoft Vi- 


sual Studio and store that package as a file (with a 
.dtsx extension) or in SQL Server’s msdb database. 
Storing SSIS packages under a well-defined folder 
structure in msdb offers a couple of advantages. Be- 
cause the SSIS packages are stored in a centralized 
location, you can easily back up all of them by back- 
ing up the msdb database. Plus, when you store SSIS 
packages in the msdb database, they’re protected by 
another layer of security. When the SSIS packages 
are stored as files, they can be accessed and modified 
by anyone who has read/write permission for the 


To avoid these hassles, I developed the SSIS Pack- 
age Management solution. As Figure 2 shows, you 
can use it to import a SSIS folder structure (and all 
its underlying SSIS packages) to msdb, export an 
SSIS folder structure from msdb, and delete an exist- 
ing SSIS folder structure from msdb. The SSIS Pack- 
age Management solution consists of three packages: 
a main package named Package_Management.dtsx 
and two subpackages named Extract_Packages_ 
From_MSDB_To_Files.dtsx and Import_SSIS_ 
Packages_To_MSDB.dtsx (see Figure 3). 

Package_Management.dtsx has a few control 
switches (i.e., package variables) that determine 
which subpackage should run and the actions to be 


Prompt for invalid 
usage format 


EA 2 


Import , export or clean »> a) 


Extract an SSIS folder 
structure and its 


Figure 2 


The SSIS Package 
Management solution’s 
functionality 


Solution Explorer 


L extract SSIS packages to File 


[© Data Sources 
[27 Data Source Views 
[E SSIS Packages 
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Figure 3 


The SSIS Package 
Management solution's 
three packages 
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ti, 
L 


packages from MSDB 


Import a SSIS folder 
structure and its 
packages to MSDB 


folders in which the packages are stored (unless they 
are password protected). 

Despite the benefits of storing SSIS packages in 
msdb, getting them into the database can be a hassle, 
especially when 
your packages are 
scattered through- 
out hundreds of 
subfolders in the 
SSIS folder struc- 
ture. In such cases, 
you need to manu- 
ally import all the 
packages, one at 
a time. Although 
you can use the 
dtutil command- 
line utility to copy, 
move, and delete 
SSIS packages, it can be tricky and time-consuming 
to use. And if there’s an existing SSIS folder struc- 
ture on msdb, it must be removed prior to importing 
a new one. 


Clean up MSDB by removing 
the existing SSIS folder 
structure and its packages 


taken. In all, there are five variables 

that you need to set to use the SSIS 

Package Management solution: 

e Root_Folder: This variable is used 
to specify the folder where the SSIS 
folder structure will be extracted 
to (when exporting from msdb) or 
imported from (when importing 
to msdb). The folder path must be 
a Universal Naming Convention 
(UNC) path. The account that’s 
running the solution must have read/write 
permission for the folder specified in Root_ 
Folder. 

e SQL_Server_Instance_Name: This variable is 
used to specify the name of the SQL Server ma- 
chine that SSIS folder structure will be extracted 
from or imported to. 

* Import_To_MSDB: Setting this variable to 1 puts 
the solution into import mode, which means the 
SSIS folder structure will be imported to msdb. 
When you don’t want to perform an import op- 
eration, you set this variable to 0. 

* Export_From_MSDB: Setting this variable to 1 
puts the solution into export mode, which means 
the SSIS folder structure will be exported from 
msdb. When you don’t want to perform an export 
operation, you set this variable to 0. 

* Just_Cleanup: Setting this variable to 1 puts the 
solution into cleanup mode, which means the 
existing SSIS folder structure will be deleted from 
msdb. (No other action is taken.) When you don’t 
want to perform a cleanup operation, you set this 
variable to 0. 
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Have any TechEd video of your own to share? Feel free to create an ITTV.net account and 


upload your own! Visit www.ITTV.net for more details. 


2009 Best of TechEd Award Winners! 
TechEd is one of the most Our team of judges examined more than 170 products from doz- a 
ens of vendors to come up with our Best of TechEd award winners, 


tech-ed 


ca | 2009 
and thousands of attendees also voted for the 2009 Best of TechEd (Punua 3334 [04 


the year, and TechEd 2009 Attendees' Pick Awards. See the winners of both awards programs at 
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The machine where the solution will be running, the 
root folder where the SSIS folder structure will be 
extracted to or imported from, and the SQL Server 
machine where the SSIS packages will be imported 
to or extracted from can all be on separate machines, 
which provides complete flexibility. 

To use the import mode, set Import_To_MSDB 
to 1, Export_From_MSDB to 0, and Just_Cleanup 
to 0. The solution will then move the SSIS folder 
structure from the location specified in Root_Folder 
to the msdb database on the server specified in SQL_ 
Server_Instance_Name. Figure 4 shows an example 
of the SQL_Server_Instance_Name and Root_ 
Folder variables set. 

To use the export mode, set Export_From_MSDB 
to 1, Import_To_MSDB to 0, and Just_Cleanup to 
0. The solution will then move the SSIS folder struc- 
ture in the msdb database on the server specified in 
SQL_Server_Instance_Name to the folder specified 
in Root_Folder. (The specified folder must already 
exist.) 

To use the cleanup mode, you set Just_Cleanup 
to 1, Import_To_MSDB to 0, and Export_From_ 
MSDB to 0. The solution will then remove the SSIS 


Script Task 


Package_Management 
Package_Management 


Package_Management 
Package_Management \\ComputerA\SHARE\test 


ServerA 


AR Properties [x] Variables EAlSolution Explorer |g dass view |i Log Events [32 Toolbox 

folder structure from msdb on the server specified in Figure 4 
SQL_Server_Instance_Name. 

As you might have noticed, only one of the 
Import_To_MSDB, Export_From_MSDB, and 
Just_Cleanup variables can be set to 1 at any given 
time. If you set more than one of these variables to 1, 
you'll receive an error message like that in Figure 5. 

You can download the SSIS Package Manage- 
ment solution from the SQL Server Magazine web- 
site by going to www.sqlmag.com, entering 101918 
in the InstantDoc ID text box, and clicking the 
101918.zip hotlink. The solution works on SQL 
Server 2005 and later. 


Setting the Package_ 
Management.dtsx 
variables 


InstantDoc ID 101918 


Invalid input values ; Import_To_MSDB , Export_From_MSDB and Just_Cleanup parameters can NOT be set to 1 at the same time - only one of these parameters 


can be set to 1 


Figure 5 


Error message received when some of the variables are set incorrectly 


How to Determine the Average 
Number of Days Between Orders or 
Other Important Events 


Use the ROVV_NUMBER function for fast queries 


lationship Management (CRM) program. One 
important Key Performance Indicator (KPI) we 
wanted to develop for the program was the average 
number of days between orders. We wanted this KPI 
to determine the customers’ usual purchase patterns 
so we could answer questions such as “Do some cus- 
tomers purchase more often than the average cus- 
tomer?” and “Is the average time between purchases 
increasing?” (An increase could be a sign that the 
customer is at risk for leaving the program.) 
Determining the average number of days between 
orders involves finding two consecutive orders from 


i *ve been working on a large B2B Customer Re- 
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the same customer and calculating the number of 

days between them. The first solution I tried used a E 

looping algorithm that worked something like this: 7 

1. Find the first order for a customer and save the 
order date in a temporary variable. 

2. Find the next order and save that order date in a 
second temporary variable. 

3. Find the difference between the two dates and 
save that value. 

4. Move the second temporary variable’s value into 
the first temporary variable, retrieve the next 
record, and save that record's order date in the 
second temporary variable. Find the difference 


Kristen Cheyney 


lead marketing analytics architect, 
Wirestone, Boise, Idaho, 


kristen.cheyney @ wirestone.com 
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@)-- Query that produces a chronological list of each customer’s orders 
SELECT CustomerKey,FullDateAlternateKey,ROW_NUMBER() OVER 
(PARTITION BY CustomerKey ORDER BY FullDateAlternateKey) 
FROM FactInternetSales FIS 
INNER JOIN DimTime DT ON DT.TimeKey=FIS.OrderDateKey 


@) -- Code that makes sure there isn't more than one order from the same customer 


-- on any given day 
SELECT CustomerKey,OrderDT,ROW_NUMBER() OVER 
(PARTITION BY CustomerKey ORDER BY OrderDT) AS OrderSequence 
INTO #orders 
FROM 
(SELECT FIS.CustomerKey 
-- Code that returns just the date part of the datetime field 


,CAST (CONVERT (varchar (19), FullDateAlternateKey, 181) AS datetime) AS OrderDT 


FROM dbo.FactInternetSales FIS 
INNER JOIN DimTime DT ON DT.TimeKey=FIS.OrderDateKey 


GROUP BY FIS.CustomerKey,CASTCCONVERT(varchar(1@), FullDateAlternateKey, 191) 


AS datetime) 
D D 


Ò -- Code that joins the #orders table to itself, matches corresponding orders, and 


-- measures the days between those orders 
SELECT Copyl.CustomerKey,Copy1.OrderDT AS CopylOrderDT, 
Copy1.OrderSequence AS CopylOrderSeq, 
Copy2.OrderDT as Copy20rderDT, Copy2.OrderSequence as Copy20rderSeq, 
CASE 
WHEN DATEDIFF(d,Copy1.OrderDT,Copy2.OrderDT)=@ THEN 1 ELSE 
DATEDIFF(d,Copy1.OrderDT,Copy2.OrderDT) END AS DaysBtwnOrders 
INTO #BtwnOrders 
FROM #orders AS Copyl 
JOIN #orders AS Copy2 
-- Code that adds 1 to the Copyl order sequence 
ON Copy1.OrderSequence+1=Copy2.OrderSequence 
AND Copyl.CustomerKey=Copy2.CustomerKey 


D SELECT CustomerKey,cast(AVG(DaysBtwnOrders)AS float) AS Avg 
FROM #BtwnOrders 
GROUP BY CustomerKey 
ORDER BY CustomerKey 


CustomerKey FullDateAlternateKey 
11000 2001-07-22 00:00:00.000 1 
11000 2003-07-22 00:00:00.000 2 
11000 2003-07-22 00:00:00.000 3 
11000 2003-11-04 00:00:00.000 4 
11000 2003-11-04 00:00:00.000 5 
11000 2003-11-04 00:00:00.000 6 
11000 2003-11-04 00:00:00.000 7 
11000 2003-11-04 00:00:00.000 8 
11001 2001-07-18 00:00:00.000 1 
11001 2003-07-20 00:00:00.000 2 
Figure 6 


Sample results from the query that produces a 


chronological list of each customer’s orders 


CustomerKey OrderDT OrderSequence 
11000 7/22/2001 1 
11000 7/22/2003 2 
11000 11/4/2003 3 
11001 7/18/2001 1 
11001 7/20/2003 2 
Figure 7 


Sample results from the code that makes sure 
there isn’t more than one order from the same 
customer on any given day 
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between the two dates and save that value. 
5. Continue looping through all the 
records for a customer, comparing the 
dates from consecutive orders until you 
encounter the next customer’s records. 


Not only were the algorithm queries com- 
plicated to write, they didn’t perform very 
well. When I tested the algorithm against 
the FactInternetSales table, it took more 
than four minutes to complete on my server 
running SQL Server 2005 SP2. Fact 
Internet Sales has only around 60,000 re- 
cords and 19,000 customers. I knew that 
the performance of this algorithm would 
only get worse as the numbers got larger. 

I decided to try a different approach. I 
began by using the ROW_NUMBER func- 
tion with an OVER clause to get a chrono- 
logical list of each customer’s orders. After 
I saved the output to a temporary table, I 
used an INNER JOIN operation to bring 
adjacent records together. Callout A in 
Listing 1 shows this query. 

As Figure 6 shows, the query produces 
a list of orders sorted by date for each 
customer and an additional column that 
specifies the sequence of each order in 
each customer list. If you look at the data 
closely, you'll see that some orders occur 
on the same day. We needed to have only 
one order per customer per day because 
we wanted to measure the average number of days 
between orders. (You might choose to do this differ- 
ently depending on the business problem and rules.) 
So, as callout B shows, I used the GROUP BY state- 
ment to get only one order per customer per day. I 
also eliminated the time portion from the datetime 
field. I then used a SELECT statement to get the re- 
sults into a temporary table named #orders. At this 
point, the results looked like that in Figure 7. 

The next step was to join the table to itself (i.e., 
join Copyl of the #orders table to Copy2 of the #or- 
ders table) in such a way that consecutive orders could 
get matched, thus making date comparisons simple. 
The idea was to join the first order to the second order, 
the second order to the third order, and so on for each 
customer, as Figure 8 shows. The trick that turned this 
idea into reality was adding 1 to the RowNumber of 
Copy] before joining it to Copy2. That way, a custom- 
er’s first order is joined to his or her second order. 

With the consecutive orders matched, I used the 
DATEDIFF function to measure the time between 
them. The beauty of this approach is that you don’t 
have to worry about the endpoints because they take 
care of themselves. The last order for a customer 
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isn't joined to any order 
(e.g., customer 11000 


Copy 1 of #Orders Table 


order 3 in Copyl isn’t 
joined to any order in 
Copy2) and is therefore 
automatically dropped 
from the result set. 
Callout C in Listing 1 
highlights the code that 
joins the #orders table 
to itself, matches cor- 


Copy 2 of #Orders Table 


CustomerkKey OrderDt 


7/22/2001 


responding orders, and 
measures the days between those orders. The result- 
ing data set is put into the new #BtwnOrders table. 
If you were to select the records from the #BtwnOr- 
ders table by adding 


SELECT * FROM #BtwnOrders 


at the end of the code 


sen 


second to run against the FactInternetSales table, 
which is a vast improvement over the four minutes 
required by the looping algorithm. When I ran this 
solution against 12 million records for 120,000 cus- 
tomers, it completed in just over two minutes. 

You can download DaysBetweenOrders.sql from 


Figure 8 


Matching consecutive 
orders to make date 
comparisons simple 


in callout C, you'd see | CustomerKey CopylorderDT CopylorderSeq Copy20rderDT Copy20rderSeq DaysBtwnOrders 
results like that in | 11000 7/22/2001 1 7/22/2003 2 730 
Figure 9. 11000 7/22/2003 2 11/4/2003 3 105 
11001 7/18/2001 1 7/20/2003 2 732 

The final step was to | 11001 7/20/2003 2 6/12/2004 3 328 
use the AVG function in | 11002 7/10/2001 1 7/4/2003 2 724 
a SELECT statement to | 11002 7/4/2003 2 8/27/2003 3 54 
calculate the average num- 
ber of days between orders for each customer. Callout the SQL Server Magazine website by going to www Figure 9 


D in Listing 1 shows this code. When you run Days 
BetweenOrders.sq] in its entirety you get results such as 


CustomerKey Avg 
11000 417 
11061 538 


I found that this solution took less than one 


Avoid Referential Integrity 


.sqlmag.com, entering 101924 in the InstantDoc ID 
text box, and clicking the 101924.zip hotlink. Using 
DaysBetweenOrders.sql as a template, you can create 
your own solution that determines the average number 
of days between important events, such as customer 
orders, server failures, support calls, or website visits. 
InstantDoc ID 101924 


Deleting Records from Databases 
Let this script or stored procedure do the work for you 


the company in which I work, we 
sometimes need to release a devel- 
opment database to production that 


contains only the schema and no data. The fastest 
and easiest way I found to do this is to back up the de- 
velopment database, create a new database from the 
backup, then use a script to delete all the records from 
the newly created database, leaving only the schema. 
When you delete a database’s records, you need to 
be careful about referential integrity errors. These er- 
rors will occur if you delete the records in tables with 
foreign keys before deleting the records in the refer- 
enced tables. To avoid this problem, I created a script, 
DELETE_RECORDS.sq], that disables all the foreign 
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keys in the database, deletes all the records, then re- 
enables all the foreign keys. You can use this generic 
script to delete all the records from any database. It 
uses the ALTER TABLE statement to disable and 
re-enable the foreign keys, so ALTER TABLE permis- 
sion for the database is needed to use the script. With- 
out this permission, you'll get a foreign key violation 
error. 

At our company, some database owners (DBOs) 
occasionally need to delete all the data from SQL 
Server Integration Services (SSIS) packages and re- 
fresh them with new data. A fellow DBA recently 
asked me whether there’s a way to delete all the re- 
cords in a database without disabling the foreign keys 


Sample results from 
running DaysBetween 
Orders.sq] in its 
entirety 


Errors When 


Chandra Sekhar 
Pathivada 


business intelligence and application 


DBA manager, Satyam Computer Services, 
sekhar@ calsql.com, www.calsql.com 


June 2009 15 


MORE SQL SERVER SOLUTIONS FROM THE FIELD 


because giving ALTER TABLE permissions to DBO 
user accounts isn’t advisable. So, I created a stored 
procedure, PR_HIERARCHIAL_DATA, that de- 
letes all the records in a database without disabling 
the foreign keys. 

To avoid referential integrity errors, PR_HIER- 
ARCHIAL_DATA determines the hierarchical order 
of all the tables in a database, then deletes the data 
from those tables in reverse hierarchical order. Iden- 
tifying the order is bit difficult because many types of 
relationships can exist in a database, including one-to- 
one, one-to-many, and many-to-many relationships. 

To identify the hierarchical order, the stored pro- 
cedure uses information in the sysreferences systems 
table and sys.objects system view. In sysreferences, 
it uses the fkeyid column, which contains the IDs 

of the referencing tables 
(i.e., the tables that con- 


We hope you're enjoying tain foreign keys), and 


the rkeyid column, which 


these Reader to Reader sonais the IDs of the 
solutions. We'd like to referenced tables. In sys. 


objects, it uses the ob- 


see your best solution. jectid column, which 


Send it to R2R@sqlmag 
.com. If we print your 


contains object IDs. 
PR_HIERARCHI- 

AL_DATA uses a recur- 

sive common table ex- 


solution, youll get $ | 00. pression (CTE) query to 
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join the sysreferences ta- 
ble multiple times based on a join between the rkeyid 
and fkeyid columns. I prefer using CTEs because they 
offer a more readable form of the derived table that 
can be declared once and referenced multiple times in 
a query. This CTE query results in a hierarchical list 
that contains the tables with foreign keys and the ta- 
bles those foreign keys are referencing. However, the 
list doesn’t include tables that don’t have any relation- 
ships (i.e., independent entities). Another CTE query 
provides that information. The results from both 
CTE queries are put into a temporary table. Dupli- 


cate table names (which result when a table references 
to two or more tables) are removed with the MAX 
function. 

The stored procedure then uses the information in 
the temporary table to delete the data in the tables 
in an order that avoids referential integrity errors. If 
PR_HIERARCHIAL_DATA finds conflicted tables 
(i.e., tables that reference each other), it will print the 
names of those tables. You will then need to manually 
delete the data from those tables. 

You can use the PR-HIERARCHIAL_ DATA 
stored procedure three ways. To simply display a 
hierarchical list of the tables with foreign keys and the 
tables those foreign keys are referencing, you execute 
the stored procedure without passing in any param- 
eter values, as in 


EXEC PR_HIERARCHTAL_DATA 


If you want to output a list of the tables in hierar- 
chical sequence so you can design a SSIS package 
or write an insert script that will load data into the 
tables, you use the INSERT parameter when you call 
the stored procedure, as in 


EXEC PR_HIERARCHIAL_DATA 'INSERT' 


If you want to delete the data from the database ta- 
bles, you use the DELETE parameter, as in 


EXEC PR_HIERARCHIAL_DATA 'DELETE' 


You can download the PR_HIERARCHIAL_ 
DATA stored procedure and DELETE_RECORDS 
.sql script from the SOL Server Magazine website by 
going to www.sqlmag.com, entering 101931 in the 
InstantDoc ID text box, and clicking the 101931.zip 
hotlink. The stored procedure and script were writ- 
ten for SQL Server 2005. You don’t need to customize 
any code before running them. 

InstantDoc ID 101931 
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CLR Stored Procedure Finds Broken 
Catch them before they catch you off guard 


hen stored procedures, user-defined func- 
W: (UDFs), views, Data Manipulation 

Language (DML) triggers, database- 
level Data Definition Language (DDL) triggers, 
and server-level DDL triggers are schema-bound, 
you can’t modify their referenced objects. However, 
that’s not the case for non-schema-bound objects. 
You can modify their referenced objects, which can 
cause the non-schema-bound objects to fail when 
executed. Failures can occur for many reasons, such 
as references to invalid column names, invalid object 
names, and invalid linked server names. 


operator in ‘SET OPTION ON’ within a function.) 
The stored procedure uses SET NOEXEC ON. When 
you use this statement, DDL and DML statements 
are compiled but not executed. Thus, you can reissue 
a CREATE statement for an object even though that 
object already exists in the database and you won't re- 
ceive an object already exists error. You will, however, 
find out whether there’s a compile or parse error. 

I wanted the BrokenObjects stored procedure 
to be able to look for broken objects in a particular 
database rather than in all the databases on a SQL 
Server instance, so I decided to pass in a database 


Jameel Ahmed 
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You might think you have schema-bound objects, 
but in reality you might not. For example, one way to 
create a schema-bound dependency is to use T-SQL’s 
SCHEMABINDING clause when you create a view 
or UDF. However, this clause works only when you 
use a two-part name (schema.object) for the refer- 
enced object. If you use a three-part name (database 
schema.object) or four-part name (linked_server 
.database.schema.object) for the referenced object, 
the view or UDF will be non-schema-bound. 

All schema-bound objects are bindable. Here’s 
what that means: When you execute T-SQL code, 
the query optimizer performs the following tasks: 
compiling (parsing and binding), optimizing (gen- 
erating an efficient execution plan), and executing. 
The parsing process checks for T-SQL language syn- 
tax errors in the T-SQL code being submitted to the 
query optimizer, without checking to see whether the 
objects exist. The binding process binds the objects’ 
names extracted from the parsing process to the ac- 
tual objects in the database to make sure the objects 
actually exist in the database. Binding only occurs 
for DML statements or DDL statements that con- 
tain DML statements (e.g., statements that create 
a view). For some objects, binding can be deferred 
until runtime, as in the case of a stored procedure 
that references an object that doesn’t exist yet. If a 
stored procedure references an existing object, then 
binding validation isn’t deferred. 

Non-schema-bound objects might be bindable, 
depending on the presence of an invalid object refer- 
ence. So, to find non-schema-bound objects that must 
be verified, you can query the sys.sql_modules system 
view, where the is_schema_bound field equals 0. 

I developed a SQL CLR stored procedure named 
BrokenObjects to identify unbindable, or broken, ob- 
jects. (I couldn’t develop a SQL CLR function because 
including SET OPTION in a function generates the 
error Invalid use of side-effecting or time-dependent 
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name as an input parameter. To change the database 
context, I had to use the command 


USE database_name 


New Project 


E- Database Projects 
L R Other Protert Tunes — = 


| A project for creating classes to use in SQL Server 


w 


| BrokenObjects 
| C:\BrokenObjects 


Name: 
Location: 


Solution Name: | Broken Ob nObjects Tii 


where database_name is the name of the database to 
check. When you run a USE command after a SET 
NOEXEC ON statement, the T-SQL scripting en- 
gine won't change the database context. It will only 
parse the USE command for validity (i.e., it will 
check to see whether the database name passed to 
the USE command exists). So, I had to put the USE 
command before the SET NOEXEC ON statement. 
After both the USE and SET NOEXEC ON state- 
ments run, the stored procedure uses a T-SQL query 
to find the broken objects, then compiles those ob- 
jects’ DDL (data definition language) statements. 
BrokenObjects uses the Microsoft .NET Frame- 
work 2.0, so you need Microsoft Visual Studio 2005 
or later to build and deploy it. Because this CLR 
stored procedure doesn’t reference any .NET 3.0 or 
.NET 3.5 Framework specific classes, you can deploy 


Figure 10 
Creating the 


BrokenObjects project 
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object_name | error message 
v_customerl Invalid column name 'NAME'. CREATE VIEW dbo.y_customerl AS SELECT id, NAME FROM dbo.customer! 


753265444 YV v_customer2 Invalid object name 'dbo.customer2'. CREATE VIEW dbo.v_customer2 AS SELECT id NAME FROM dbo.customer2 
1920021356 Y v_test2 Invalid object name ‘dbo. test2". CREATE VIEW dbo.v_test2 AS SELECT id NAME FROM dbo.test2 


Figure I 1 


Sample results from BrokenObjects 


EI Common Assembly.dbo.sp broken objects 'tempdb', 

| 

s — = ll ee 
75777 Sl "ary pee | 

| EJ Results| Cà Messages | UU 

Parsing ... [dbo].[up_sql_mod] 

Parsing ... [dbo].[up_test] 

Parsing ... [dbo].[v_customerl] 

Parsing ... [dbo]. [v_customer2] 

Parsing ... (dbo). [v_customer3] 

Parsing ... [dbo]. [v_sql_mod] 


(3 row(s) affected) 


Figure 12 


Sample message when you set the last parameter to | 


it on SQL Server 2008 and SQL Server 2005. 

To build and deploy the BrokenObjects stored 
procedure, follow these steps: 

1. Download the CLR stored procedure’s source 
code (BrokenObjects.cs) by going to the SOL 
Server Magazine website (www.sqlmag.com), 
entering 101939 in the InstantDoc ID text box, 
and clicking the 101939.zip hotlink. 

2. Using SQL Server Management Studio (SSMS), 
connect to a SQL database server and create a 
database named Common_Assembly by running 
the command 


CREATE DATABASE Common_Assembly 


This is where you'll be deploying the CLR stored 
procedure to. 

3. Open Visual Studio. On the File menu, select 
New, then Project Solution. In the New Proj- 
ect dialog box that Figure 10, page 17, shows, 
navigate to Visual C#, Database, SQL Server 
Project in the Project types tree. In the Name text 
box, enter BrokenObjects. The Solution Name 
field will automatically populate as you enter the 
project name. Change the Location field to C:\ 
BrokenObjects. Click OK. 

4. When the Add Database Reference dialog box 
appears, click Add a New Reference. In the 
New Database Reference dialog box, specify the 
Server Name and Common_Assembly as the 
database. 

5. Create a strong name key file for signing the as- 
sembly. On the Project menu, select Broken 
Objects Properties. When the Project Designer 
appears, click the Signing tab. After making 
sure the Sign the assembly check box is selected, 
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choose New from the drop-down menu. In the 
Create Strong Name Key dialog box, enter Bro- 
kenObjects for the filename, clear the Protect my 
key file with password check box, and click OK. 

6. Add the stored procedure. On the Project menu, 
select Add Stored Procedure. Set the name to 
BrokenObjects.cs. Enter the source code for 
BrokenObjects.cs. 

7. On the Project menu, select Build Solution. 

8. On the Project menu, select Deploy Solution to in- 
stall the assembly and create the stored procedure. 


You can now deploy the SQL CLR procedure to 
other servers. You can even deploy it to servers that 
don’t have Visual Studio by using the compiled .NET 
assembly (which is in C:\BrokenObjects\Broken 
Objects\BrokenObjects\bin\Debug\BrokenObjects. 
dil) and the Install_Assembly.sql and Uninstall_As- 
sembly.sql scripts (which are in the 101939.zip file). 

After BrokenObjects is deployed, you can use 
Test.sql in the 101939.zip file to test the installed 
assembly. This script creates a schema-bound view 
and a referenced table in the Common_Assembly 
database, then tries to change the name of one of 
the table’s columns. Next, the script creates a non- 
schema-bound view and a referenced table, then tries 
to change the name of one of the table’s columns. 
With the stage set, the script then runs the BrokenOb- 
jects stored procedure. Figure 11 shows the results. 

To run the BrokenObjects stored procedure 
against one of your databases, follow the syntax 


Common_Assembly.dbo.sp_broken_objects 
'DB', 1 


where DB is the name of the database you want to 
check for unparsable objects. The last parameter, if 
set to 1, will call the SqlPipe.Send .NET method. If 
it’s set to 0, the method isn’t called. This method is 
used to display messages similar to that of a PRINT 
statement. The message will specify which object it’s 
attempting to parse, as Figure 12 shows. 

Note that the BrokenObjects stored procedure’s 
results are dependent on the behavior of SET NO- 
EXEC ON. There might be situations in which SET 
NOEXEC ON doesn’t find every binding error, as 
Scenario 3 in Test.sql demonstrates. I haven't tested 
the stored procedure with all the various types of 
UDFs. Once again, the results will depend on the be- 
havior of the SET NOEXEC ON being performed 
on the object’s DDL statement. SQL 

InstantDoc ID 101939 
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Use cursors for large partitions 


unning aggregations are calculations that 

continue to accumulate measures over a 

sequence—typically temporal, and possibly 
within partitions. An example of obtaining a running 
aggregate is finding a running sum of quantity for each 
employee and month, from a table that holds a row for 
each employee and month with measures such as quan- 
tity and value. That is, for each employee and month, 
you return the sum of all quantities from the beginning 
of the employee’s activity until the current month. 

Last month I started a series of articles that explore 
the performance aspects and algorithmic complexity of 
different solutions to running aggregations. I covered 
two set-based solutions—one using subqueries and 
another using joins. (See “Subqueries and Joins for Run- 
ning Aggregates,” May 2009, InstantDoc ID 101623.) 
This month I cover a solution based on cursors. 

Some developers and database administrators 
consider cursors to be evil, even going so far as to rec- 
ommend avoiding them at all costs. Indeed, set-based 
solutions have many advantages over cursor-based 
solutions in terms of being more aligned with the rela- 
tional model, requiring less code, being more readable, 
involving less maintenance, and typically performing 
better. However, cursors perform better than set-based 


LISTING |: DDL Statement to Create 


Sales Table 


SET NOCOUNT ON; 
USE tempdb; 


IF OBJECT_ID('dbo.Sales', 
Sales; 


CREATE TABLE dbo.Sales 
¢ 


"U') IS NOT NULL DROP TABLE dbo. 


solutions in some cases. In this article I show that 
under certain circumstances, cursor-based solutions to 
running aggregations provide better performance than 
set-based solutions. 


Sample Data 

In this article I use the same Sales table as last month. 
Run the code in Listing 1 to create the table. The Sales 
table contains a row for each employee and month. 
The definition of the Sales table contains an attribute 
for the employee ID (empid), month (dt), quantity 
(qty), and value (val). The primary key (as well as the 
clustered index) is defined on the key list (empid, dt). 
The clustered index follows the indexing guidelines 
to support solutions to running aggregates—the key 
list should be the partitioning column(s) 
followed by the ordering column(s), and the 
aggregated measures should be covered by the 
index as well. 

Run the code in Listing 2 to create the 
helper function GetNums, which returns a table of 
numbers with a requested number of rows. Next, run 
the code in Listing 3, page 20, to populate the Sales 
table with the desired number of partitions (employees) 
and the desired partition size (rows per employee). 
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LISTING 2: DDL Statement to Create 


GetNums Function 


IF OBJECT_ID(‘dbo.GetNums’ , 
DROP FUNCTION dbo.GetNums ; 


AS 
RETURN 
WITH 
LØ  AS(SELECT 1 AS 


‘IF’) IS NOT NULL 


CREATE FUNCTION dbo.GetNums(@n AS BIGINT) RETURNS TABLE 


c UNION ALL SELECT 1), 

c FROM LØ AS A CROSS JOIN LØ AS B), 
L2 AS(SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 AS B), 

c FROM L2 AS A CROSS JOIN L2 AS B), 

c FROM L3 AS A CROSS JOIN L3 AS B), 


empid INT NOT NULL, -- partitioning Lir  AS(SELECT 1 AS 

column 

dt DATETIME NOT NULL, -- ordering column L3: AS(SELECT 1 AS 

qty INT NOT NULL DEFAULT (1), -- measure 1 L4  AS(SELECT 1 AS 

val MONEY NOT NULL DEFAULT (1.98), -- measure 2 L5 AS(SELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B), 


CONSTRAINT PK_Sales PRIMARY KEY(empid, dt) 
DE 
GO 
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Nums AS(SELECT ROW_NUMBER() OVERCORDER BY (SELECT @)) AS n FROM L5) 


SELECT n FROM Nums WHERE n <= @n; 
GO 
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LISTING 3: Code to Populate Sales Table with stored, as well as a few local variables where the values 
Sample Data from each row will be stored. The code declares a 
DECLARE cursor based on a query that sorts the data by empid 

G titi AS INT, i à 
aree en DA AS INT, and dt. Based on this ordering, the code fetches the 
Cogan AS DATETIME: cursor records one at a time. In each iteration of the 
SET @num partitions = 19000; loop, the code handles the current row. If the current 
SET @rows_per_partition = 19; p A $. 48 á š 
SET @start_dt = ‘20099101’; row’s empid value is different than the previous row’s 
TRUNCATE TABLE dbo.Sales; value, this indicates that the current row is the first in 
INSERT INTO dbo.Sales WITH (TABLOCK) (empid, dt) the partition (for the employee). In such a case, the 
SELECT NP.n AS empid, DATEADD(day, RPP.n - 1, @start_dt) AS dt code initializes the variable holding the current running 

FROM dbo.GetNums(@num_partitions) AS NP F 

CROSS JOIN dbo.GetNums(@rows_per_partition) AS RPP; aggregate (@sumqty) with 0. The code then adds the 


current quantity to @sumqty and inserts a row with 

the current details and the running sum into the table 

As a basis for comparison, I'll use last months variable @Result. Finally the code queries the table 
set-based solution using subqueries, which Listing 4 variable to return the result. 


shows. The query in Listing 4 returns a running sum The clustered index created on the attributes 

of quantity for each employee and month. (empid, dt) as the key list is ideal for this solution. 

Figure 1 shows the execution plan for the query that 

Cursor-Based Solution the cursor is based on. As you can see in the plan, SQL 
Listing 5 shows the cursor-based solution for our run- Server performs an ordered scan of the index. 

ning sum request. Like I did last month, I'll now analyze the solu- 

LISTING 4: Set-Based . The solution is  tion's algorithmic complexity to see how changing 

Solution Using Subqueries_ .. ) en . É j 

fairly straightfor- various factors in the solution affects its performance. 

SELECT empid, dt, qty, ward. The code The factors that I'll examine are the number of 

(SELECT SUM(S2.qty) i : : 

FROM dbo.Sales AS S2 defines a table aggregations to be calculated, the number of parti- 

NEE S s NN variable called tions involved, and the average number of rows per 

FROM dbo.Sales AS S1; @Result where partition (assuming even distribution for the sake of 


the result will be simplicity). Again, I'll use the letter p to represent 


LISTING 5: Cursor-Based Solution 


DECLARE @Result AS TABLE 
enpi Query 1: Query cost (relative to the batch): 100% 
pid INT, p» A . 
dt DATETIME, SELECT empid, dt, qty FROM dbo.Sales ORDER BY empid, dt; 
qty INT, 
sumqty BIGINT E) bo 
JE Clustered Index Scan (Cluster. 
SELECT 1 1 
DECLARE Cost: 0 % [Sales]. [PK_Sa les] 
@empid AS INT, ` Cost: 100 % 
@prvempid AS INT, 
@dt AS DATETIME, Clustered Index Scan (Clustered) 
@qty AS INT, Scanning a clustered index, entirely or only a range. 
@sumqty AS BIGINT; 
Physical Operation Clustered Index Scan 
PERRE CURSOR FES eats FOR Logical Operation Clustered Index Scan 
ea She Actual Number of Rows 100000 
FROM dbo.Sales = 2.72387 
ORDER BY empid, dt; Estimated I/O Cost a 
Estimated CPU Cost 0.990157 
OPEN C Estimated Number of Executions 1 
Number of Executions 1 
FETCH NEXT FROM C INTO @empid, @dt, @qty; Estimated Operator Cost 3.71402 (100%) 
À : Estimated Subtree Cost 3.71402 
SELECT @prvempid = Gempid, @sumqty = 9; Estimated Number of Rows 900000 
Estimated Row Size 23 B 
WHILE @@fetch_status = Ø Actual Rebinds 0 
oe eee Actual Rewinds 0 
lempid <> @prvempid Ordered T 
SELECT @prvempid = @empid, @sumqty = 9; meee rue 
Node ID 0 
SET @sumqty = @sumqty + @qty; 
Object 
INSERT INTO @Result VALUES(Gempid, @dt, @qty, @sumqty) ; [tempdb].[dbo].[Sales].[PK_Sales] 
i Output List 
FETCH NEXT FROM C INTO @empid, @dt, @qty; [tempdb] {dbo}.{Sales).empid, [tempdb].[dbo].[Sales].dt, 
END [tempdb].[dbo].{Sales].qty 
CLOSE C; 
DEALLOCATE C; 
Figure | 


SELECT * FROM GResult; 
Execution plan for query used in cursor-based solution 
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the number of partitions, r to represent the average 
number of rows per partition, and a to represent the 
number of aggregations. 

Let's express the number of rows processed by 
the cursor-based solution when one aggregation is 
involved. As you can see in the plan that Figure 1 
shows, the rows from the Sales table are scanned once 
from the clustered index, in index order. You can 
therefore express the number of rows processed as pr 
rows (again, assuming fairly even distribution of rows 
among partitions). 

Note that individual row processing is more 
expensive when you process data with a cursor than 
when you use a set-based solution. For example, if 
you represent the cost of processing n rows with a 
set-based solution as n, you can express the cost of 
processing n rows with a cursor as n(1+ o) or n + no, 
where o represent the extra overhead associated with 
the processing of a row with a cursor. In terms of run 
time, n + no typically translates to several dozens of 
times longer than just n. So if pr is the number of 
rows processed by the cursor-based solution, and 
you want to add the cursor overhead to the formula 
representing the solution’s cost, you can express it as 
pr + pro. This is in comparison with set-based code 
that just scans pr rows, and whose cost you express 
as pr. 


Effect of Number of 
Aggregates 

Last month I covered in detail the set-based solution 
using subqueries, which Listing 4 shows. As you might 
recall, I expressed the number of rows to be processed 
by this solution as ap(r + 1°)/2. I explained that unfor- 
tunately SQL Server’s optimizer performs a separate 
scan of the data for each subquery, and if you have 
a aggregates to calculate, this means that the work 
involved in handling each aggregate will occur a times. 
In other words, in terms of changes in a, this solution 
has linear complexity. 

Now let’s explore the effect that increasing the 
number of aggregations has on the cursor-based solu- 
tion. The two main factors contributing to the cost of 
the cursor-based solution are the number of rows being 
processed (pr) and the cursor overhead associated 
with each record manipulation (o). These two factors 
remain constant when the only change is in the number 
of aggregations to calculate (a). In other words, with 
respect to changes in a, the cursor solution has con- 
stant complexity, or close to it. Of course, a few more 
CPU cycles are required to calculate the additional 
aggregates, and a bit more space is necessary for the 
expanded row in the table variable, but those shouldn't 
have a dramatic effect on the solution’s performance. 
Therefore, theoretically, adding aggregations to the 
cursor solution should cause only slight performance 
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LISTING 6: Cursor-Based Solution with 
Multiple Aggregations 


DECLARE @Result AS TABLE 
C 


empid INT, 

dt DATETIME, 
qty INT, 

val MONEY, 


sumqty BIGINT, 
avgqty NUMERIC(12, 2), 
sumval MONEY, 
avgval MONEY 
); 


DECLARE 
@empid AS INT, 
@prvempid AS INT, 


@dt AS DATETIME, 
@qty AS INT, 
@val AS MONEY, 


@sumqty AS BIGINT, 
@sumval AS MONEY, 
@count AS INT; 


DECLARE C CURSOR FAST_FORWARD FOR 
SELECT empid, dt, qty, val 
FROM dbo.Sales 
ORDER BY empid, dt; 


OPEN C 

FETCH NEXT FROM C INTO @empid, @dt, G@qty, @val; 

SELECT @prvempid = @empid, @sumqty = Ø, @sumval = Ø, @count = Q; 
WHILE @@fetch_status = 9 

BEGIN 


IF @empid <> @prvempid 
SELECT @prvempid = @empid, @sumqty = Ø, @sumval = Ø, @count = Q; 


SELECT 
@sumqty = @sumqty + @qty, 
@sumval = @sumval + @val, 
@count = @count + 1; 


INSERT INTO @Result(empid, dt, qty, val, sumqty, avgqty, sumval, avgval) 
VALUES (@empid, @dt, @qty, @val, 
@sumqty, 1.*@sumqty / @count, @sumval, @sumval / @count); 


FETCH NEXT FROM C INTO @empid, @dt, @qty, @val; 
END 


CLOSE C; 
DEALLOCATE C; 


SELECT * FROM @Result; 


degradation. 

To test this theory, I used the code in Listing 6 to 
calculate four running aggregates instead of one. The 
effect on performance was only about 10 percent deg- 
radation, thus proving the theory. 


Effect of 
Number of 
Partitions 

The next factor to eval- 
uate is the number of 
partitions involved—that 
is, the effect of increasing 
the number of partitions 
on the solution’s perfor- 
mance. Regarding the 
set-based solution, when 
calculating a single aggre- 
gate, you can express its 


Individual row 
processing is more 
expensive when you 
process data with a 
cursor than when 
you use a set-based 
solution. 
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cost as p(r + r?)/2. If you increase the number of parti- 
tions by a factor of f; you get pf(r + 1°)/2. This means 
that with respect to the number of partitions, the set- 
based solution has linear complexity; in other words, 
if you increase the number of partitions by a factor of 
J. the cost (which translates to run time) increases by a 
factor of fas well. 

Earlier I expressed the cost of the cursor solution 
as pr + pro. If you increase the number of partitions 
by a factor of f, you get pfr + pfro, meaning that with 
respect to changes in the number of partitions, the 
cursor solution should also have liner complexity. 
Figure 2 shows a benchmark in which I started with 


10,000 partitions with a constant partition size of 
10, and I kept increasing the number of partitions by 
10,000 every time until I reached 100,000 partitions. 
This benchmark clearly shows that the cursor solution 
also has linear complexity with respect to the number 
of partitions. 


Effect of Partition Size 

It’s interesting to see in Figure 2 that the cursor solu- 
tion is significantly slower than the set-based solution 
when dealing with a very small partition size (10 rows 
in this case). Next, let’s evaluate the effect of increasing 
the partition size on the solution. 

Regarding the set-based solution, if you increase 
the partition size by a factor of f; you can express the 
effect on the cost of the solution as p(rf + (rf)’)/2. This 
tells you that with respect to the partition size, the 
set-based solution has close to quadratic complexity. 
If the partition size increases by a factor of f the cost 
increases by almost f. For example, if you increase the 
partition size by 2, the run time increases by almost 4 
times. 

As for the cursor solution, when you increase the 
partition size by a factor of f, the effect on the cost is 
prf + prfo. This means that with respect to the partition 
size, the cursor solution has linear complexity. 

To test this theory, I ran a benchmark in which I 
started with 1,000 partitions with a partition size of 
100, and I gradually increased the partition size by 
100 each time until I reached a partition size of 1,000. 
Figure 3 shows the result of the benchmark. Even 
though the result merely confirms the theory, I find 
it fascinating. You can clearly see that there is a point 
where the cursor solution becomes faster than the set- 
based solution. In my benchmark, this point is around 
500 rows per partition. 


A Necessary Evil 
Perhaps you’ve grown used to the idea that cursors are 
evil. I must say that in many respects, I agree. However, 
the way SQL Server’s optimizer currently handles set- 
based solutions to running aggregates is with quadratic 
complexity (n?) with respect to partition size, whereas it 
handles cursor-based solutions with linear complexity. 
So in terms of performance, when dealing with small 
partition sizes—as many as a few hundred rows per 
partition—you're better off with a set-based solution. 
However, when dealing with large partitions—more 
than 500 rows—you'e better off with a cursor-based 
solution. In addition, adding aggregates has only a 
minor effect on the performance of a cursor-based 
solution, whereas adding partitions causes linear per- 
formance degradation. 
Next month I'll explore even more solutions to 
running aggregates. Stay tuned! SOL 
InstantDoc ID 101736 
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SQL Sera g 2008 
Development 
and Debu gging Features 


Debug I-SQL without having to use Visual Studio 


ability to debug T-SQL without using Visual 

Studio (VS) is one of the best features in 
SQL Server 2008. SQL Server 2000’s Query Analyzer 
includes the ability to debug T-SQL. However, the 
feature wasn’t included in SQL Server 2005’s SQL 
Server Management Studio (SSMS) and Query 
Editor. Although the move to SSMS in SQL Server 
2005 was well accepted, the only way to debug T-SQL 
scripts in SQL Server 2005 is by using VS. In spite 
of all the talk about the crossover of developers and 
DBAs, this wasn’t a popular move because many 


É or many DBAs and database developers the 


IntelliSense will display a stored procedure’s return 
values and all parameters in a tool tip format when 
you hover your mouse over a stored procedure’s name. 
Incomplete T-SQL syntax is displayed using a red 
squiggly marker at the end of each T-SQL statement. 
Hovering your mouse over the words underlined in 
red will display a pop-up tip that describes the error 
condition, as Figure 2, page 24, shows. 

Although very useful, IntelliSense isn’t completely 
full featured because it doesn’t perform T-SQL 
statement completion for all T-SQL statements and 
display column names for SELECT statements. That 


Michael Otey 


(motey@sqlmag.com) is technical director 
for Windows IT Pro and SQL Server Magazine 
and author of Microsoft SQL Server 2008 New 


DBAs don’t like to be forced to 
use VS. Another missing feature in 
SQL Server 2005 was T-SQL Intel- 
liSense. IntelliSense was originally 
planned for SQL Server 2005 but 
dropped from the final product. SQL 
Server 2008’s Query Editor provides 
a much more integrated develop- 
ment experience with the addition of 
T-SQL IntelliSense, code outlining, 
and debugging. 


T-SQL IntelliSense 
SQL Server 2008 IntelliSense pro- 
vides T-SQL syntax checking, auto- 


Ready 


Fle Edt View Query Project 


R. Microsoft SQL Server Management Studio 


Debug Tools Window Community Help 


‘Dungen | Oy | D Da Ea | D | 27 id 3 | 4 Ç 
M) 232 | master 


=| ? Execute D 88 v aaura CEG) = < | E ALS 


J use AdventureVorks2008 


SQLQuery1.sql -...istrator (61))*]"_ instnwnd.sql- ...nistrator ($8))* _ | 


Select * from HumanResources.| 


Dib: 


matic prompting for database object 
names, parameter completion, and help with common 
T-SQL statements. IntelliSense works automatically 
when you begin entering T-SQL statements into Query 
Editor. Figure 1 shows IntelliSense’s auto object name 
completion feature. You can see in Figure 1 how Intel- 
liSense’s auto completion feature displays a list of 
the tables and views that are available for the Human 
Resources schema in the AdventureWorks2008 data- 
base. You select entries in the drop-down list by 
scrolling to the desired object name and pressing 
either Tab or Enter. One nice feature is the fact that 
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said, there are a couple of ways you can coax Query 

Editor’s IntelliSense into displaying column names. 
One way is to first enter the clause 

FROM <schema>.<table> AS Alias 

and then go back and enter 


SELECT Alias. 


which will display a list of column names. 


Figure | 


IntelliSense’s object 
name completion 
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Debug I-SQL without having to use Visual Studio 


ability to debug T-SQL without using Visual 

Studio (VS) is one of the best features in 
SQL Server 2008. SQL Server 2000’s Query Analyzer 
includes the ability to debug T-SQL. However, the 
feature wasn’t included in SQL Server 2005’s SQL 
Server Management Studio (SSMS) and Query 
Editor. Although the move to SSMS in SQL Server 
2005 was well accepted, the only way to debug T-SQL 
scripts in SQL Server 2005 is by using VS. In spite 
of all the talk about the crossover of developers and 
DBAs, this wasn’t a popular move because many 


É or many DBAs and database developers the 


IntelliSense will display a stored procedure's return 
values and all parameters in a tool tip format when 
you hover your mouse over a stored procedure’s name. 
Incomplete T-SQL syntax is displayed using a red 
squiggly marker at the end of each T-SQL statement. 
Hovering your mouse over the words underlined in 
red will display a pop-up tip that describes the error 
condition, as Figure 2, page 24, shows. 

Although very useful, IntelliSense isn’t completely 
full featured because it doesn’t perform T-SQL 
statement completion for all T-SQL statements and 
display column names for SELECT statements. That 
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DBAs don’t like to be forced to 
use VS. Another missing feature in 
SQL Server 2005 was T-SQL Intel- 
liSense. IntelliSense was originally 
planned for SQL Server 2005 but 
dropped from the final product. SQL 
Server 2008’s Query Editor provides 
a much more integrated develop- 
ment experience with the addition of 
T-SQL IntelliSense, code outlining, 
and debugging. 


T-SQL IntelliSense 
SQL Server 2008 IntelliSense pro- 
vides T-SQL syntax checking, auto- 
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matic prompting for database object 
names, parameter completion, and help with common 
T-SQL statements. IntelliSense works automatically 
when you begin entering T-SQL statements into Query 
Editor. Figure 1 shows IntelliSense’s auto object name 
completion feature. You can see in Figure 1 how Intel- 
liSense’s auto completion feature displays a list of 
the tables and views that are available for the Human 
Resources schema in the AdventureWorks2008 data- 
base. You select entries in the drop-down list by 
scrolling to the desired object name and pressing 
either Tab or Enter. One nice feature is the fact that 
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said, there are a couple of ways you can coax Query 

Editor’s IntelliSense into displaying column names. 
One way is to first enter the clause 

FROM <schema>.<table> AS Alias 

and then go back and enter 


SELECT Alias. 


which will display a list of column names. 


Figure | 


IntelliSense’s object 
name completion 
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Statement completion option. You 
can also use the Tools, Options, 
Text Editor, Transact-SQL, Intel- 
liSense tab to enable and disable 


Object Explorer + Rx 


_/$QLQuery 1.sql -...istrator (61))* |" instrwind.sal - ...nistrator (58))* 


Connect ~ | 32 42 @ Y Z Suse AdventureVorks2008 


E L $Q12008-2 (SQL Server 10.0.1600 - SQL2008-2\Administrator) | 


Select * from HumanResources.Employee 


exec Adventurevorks2 008 . dbo. getdeptn| 
Could not find stored procedure ‘AdventureWorks2008,dbo,getdeptn’, 


Figure 2 


Alternatively, you can fully qualify the column names 
in the SELECT statement, which lets IntelliSense dis- 
play the column names. For instance, if you enter 


Pop-up tip describing 
error in T-SQL 
statement 


SELECT <schema>.<table>. 


IntelliSense will list the column names for the table. 
There are some places where IntelliSense doesn’t 
work. For example, IntelliSense isn’t active when 
there’s a code error above the cursor or when the 
cursor is in a line that’s in a comment or within a 
literal. It also doesn’t work for encrypted database 
objects. In addition, IntelliSense won’t work if 
the script you're editing is larger than the default 


IntelliSense, as well as to enable 
and disable the underlining of 
errors and outlining parameters. 


T-SQL Code 
Outlining 
+|| Code outlining is a new feature in 
the SQL Server 2008 Query Editor 
that can make your code more 
readable. It’s especially useful for 
long stored procedures and T-SQL batches. The code 
outlining feature adds a plus sign (+) to the beginning 
of a block of code, which you can click to expand 
or collapse the code within that block. The code 
outlining feature works with the following T-SQL 
code groups: 

e Batches—T-SQL batches are ended using the GO 
keyword. All the code from the start of the file to 
the first GO statement is considered one batch. If 
there’s no GO keyword, then the entire script is 
considered a batch. There’s one batch from each 
GO statement to either the next GO statement or 
the end of the script. 

e Blocks—T-SQL blocks are marked using the 

following keywords: BEGIN - 


n5 4 


Ra Microsoft SQL Server Management Studio 
Ele Edt Wew Query Project Debug Tools Window Community Help 
A smer D |D D S/B |G ARa 
-| teate D n V 33 [3] 7 21 REO) = 2 | |^ 


J SQULQuery1.sai-...lstrator (61))* nF ~nistrator (58))* | 


END, BEGIN TRY - END TRY, 
and BEGIN CATCH - END 
CATCH. The code outlining 
feature collapses the entire block. 


drop table "dbo"."Suppliers" 
GO 
Œ if exists 
Go 
CREATE TABLE "Employees" (|...) 
GO 
CREATE 
GO 
CREATE 
Go 


(select * from sysobjects where id = 


INDEX "LastName" “EmployeelD" 

“LastName” nvarchar (20) NOT NULL , 
"FirstName" nvarchar (10) NOT NULL , 
"Tile" nvarchar (30) NULL , 


“etter nvarchar (25) MAL 


“BirthDate’ 
"HireDate” "datetime" NULL , 
"Address" nvarchar prl NULL , 


E Ò SQL Server Agent 
INDEX "PostalCoda 


CREATE TABLE "Categories" 
"CategoryID” "int" IDH 
"CategoryName" nvarch: 
"Description" "ntext" 
“Picture” "image" NUL 
CONSTRAINT "PK_Categor 


"CategoryID” 


GO 
CREATE 
GO 


INDEX "CategoryNa 


EQ CREATE TABLE "Customers" 


"Cuernmer Th” nchar (S) 


maximum size of 1MB. 

You can change the IntelliSense default values and 
turn IntelliSense on and off by navigating to SSMS's 
Tools, Options, Text Editor, Transact-SQL, General, 


Figure 3 


Code outlining feature 
in SQL Server 2008’s 
Query Editor 
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object_id('dbo.Employees') 


"int" IDENTITY (1, 1) NOT NULL , 


e Multiline statements—Multiline 
statements are T-SQL statements 
that continue for two or more 
lines in the Query Editor window. 
The outlining feature lets you 
collapse these lines back to the 
first line. 


You can see an example of the 
code outlining feature in Figure 
3. 

Code blocks hidden by code 
outlining are indicated by gray 
ellipses. Positioning your mouse 
over the ellipsis displays the 
hidden code in a pop-up tool tip, 
as Figure 3 shows. Code outlining 
is also available for SQL Server 
Analysis Services MDX queries. 


T-SQL Debugging 
SQL Server 2008 provides full-featured debugging 
capabilities that are integrated directly into Query 
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R. Microsoft SQL Server Management Studio 

Ele Edit Yew Query project Debug Tools Window Community Help 
iT New Query | D | D D E | Ds | 27 Wo |S 
: 02 V | master 


z| è Execute > my 39.27 [I] 37 My | 65 [63]& | = < | # | Ab g 


Statement completion option. You 
can also use the Tools, Options, 
Text Editor, Transact-SQL, Intel- 
liSense tab to enable and disable 


Object Explorer + Rx 


_/$QLQuery 1.sql -...istrator (61))* |" instrwind.sal - ...nistrator (58))* 


Connect ~ | 32 42 @ Y Z Suse AdventureVorks2008 


E L $Q12008-2 (SQL Server 10.0.1600 - SQL2008-2\Administrator) | 


Select * from HumanResources.Employee 


exec Adventurevorks2 008 . dbo. getdeptn| 
Could not find stored procedure ‘AdventureWorks2008,dbo,getdeptn’, 


Figure 2 


Alternatively, you can fully qualify the column names 
in the SELECT statement, which lets IntelliSense dis- 
play the column names. For instance, if you enter 


Pop-up tip describing 
error in T-SQL 
statement 


SELECT <schema>.<table>. 


IntelliSense will list the column names for the table. 
There are some places where IntelliSense doesn’t 
work. For example, IntelliSense isn’t active when 
there’s a code error above the cursor or when the 
cursor is in a line that’s in a comment or within a 
literal. It also doesn’t work for encrypted database 
objects. In addition, IntelliSense won’t work if 
the script you're editing is larger than the default 


IntelliSense, as well as to enable 
and disable the underlining of 
errors and outlining parameters. 


T-SQL Code 
Outlining 
+|| Code outlining is a new feature in 
the SQL Server 2008 Query Editor 
that can make your code more 
readable. It’s especially useful for 
long stored procedures and T-SQL batches. The code 
outlining feature adds a plus sign (+) to the beginning 
of a block of code, which you can click to expand 
or collapse the code within that block. The code 
outlining feature works with the following T-SQL 
code groups: 

e Batches—T-SQL batches are ended using the GO 
keyword. All the code from the start of the file to 
the first GO statement is considered one batch. If 
there’s no GO keyword, then the entire script is 
considered a batch. There’s one batch from each 
GO statement to either the next GO statement or 
the end of the script. 

e Blocks—T-SQL blocks are marked using the 

following keywords: BEGIN - 


n5 4 


Ra Microsoft SQL Server Management Studio 
Ele Edt Wew Query Project Debug Tools Window Community Help 
A smer D |D D S/B |G ARa 
-| teate D n V 33 [3] 7 21 REO) = 2 | |^ 
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END, BEGIN TRY - END TRY, 
and BEGIN CATCH - END 
CATCH. The code outlining 
feature collapses the entire block. 


drop table "dbo"."Suppliers" 
GO 
Œ if exists 
Go 
CREATE TABLE "Employees" (|...) 
GO 
CREATE 
GO 
CREATE 
Go 


(select * from sysobjects where id = 


INDEX "LastName" “EmployeelD" 

“LastName” nvarchar (20) NOT NULL , 
"FirstName" nvarchar (10) NOT NULL , 
"Tile" nvarchar (30) NULL , 


“etter nvarchar (25) MAL 


“BirthDate’ 
"HireDate” "datetime" NULL , 
"Address" nvarchar prl NULL , 


E Ò SQL Server Agent 
INDEX "PostalCoda 


CREATE TABLE "Categories" 
"CategoryID” "int" IDH 
"CategoryName" nvarch: 
"Description" "ntext" 
“Picture” "image" NUL 
CONSTRAINT "PK_Categor 


"CategoryID” 


GO 
CREATE 
GO 


INDEX "CategoryNa 


EQ CREATE TABLE "Customers" 


"Cuernmer Th” nchar (S) 


maximum size of 1MB. 

You can change the IntelliSense default values and 
turn IntelliSense on and off by navigating to SSMS's 
Tools, Options, Text Editor, Transact-SQL, General, 


Figure 3 


Code outlining feature 
in SQL Server 2008’s 
Query Editor 
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object_id('dbo.Employees') 


"int" IDENTITY (1, 1) NOT NULL , 


e Multiline statements—Multiline 
statements are T-SQL statements 
that continue for two or more 
lines in the Query Editor window. 
The outlining feature lets you 
collapse these lines back to the 
first line. 


You can see an example of the 
code outlining feature in Figure 
3. 

Code blocks hidden by code 
outlining are indicated by gray 
ellipses. Positioning your mouse 
over the ellipsis displays the 
hidden code in a pop-up tool tip, 
as Figure 3 shows. Code outlining 
is also available for SQL Server 
Analysis Services MDX queries. 


T-SQL Debugging 
SQL Server 2008 provides full-featured debugging 
capabilities that are integrated directly into Query 
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Editor. You can start a debugging session by clicking 
the green arrow icon, by selecting the Debug, Start 


TABLE | T-SQL Debugger 
Keyboard Shortcuts 


Debugging option from the Query Editor menu, or Keyboard Shortcut Description 
by pressing Alt+F5. (For a complete list of the Query Alt+5 Start or continue debugging 
Editor debugging keyboard shortcuts, see Table 1.) - - 
Figure 4 shows SQL Server 2008’s T-SQL debugging Shift+F5 Stop debugging 
capability. Ctrl+F10 Run to cursor 
After the debugging session has started, a yellow | F9 Toggle Breakpoint 
arrow indicates the currently executing line. In fq Step Into 
addition, the Locals window displays the value of all FLO Step Over 
the variables in the current batch and a Call Stack 
window displays the current call stack. A Debug Shift+F11 Step Out of 
toolbar is also displayed in the Query Editor menu | Ctrl+Alt+B Display breakpoints window 
and offers the Run, Pause, Stop, Show Next State- Ctrl+Alt+C Display the Cal Stack window 
ment, Step Into, Step Over, and Step Out options. | GHAR Display the Threads window 
You'll find the Debug toolbar on the far right side = . = 
of the Query Editor toolbar. The Locals window, Giles Dishlay tte duickwatchwindow 
which is shown in Figure 4, displays the local T-SQL Ctrl+Alt+0 Display the Output window 
Kx (Debugging) - Microsoft SQL Server Management Studio 
Ele Edt wew Query Project Debug Tools Window Community Help 
i Zà New Query | D | zò D E | Gs | 27 ed |S 
iw w] tete > om v 35 [7 S| DEO = Sl IN Sie o| at Z 
SQ Queryi.sql-...istrator (61))*_instnwnd.sql - ...8)) Debugging... | Seay Solution Explorer -1x 
E DECLARE @device directory NVARCHAR (520) = C3 Solution 'Solutiont! (0 projects) 
Ü SELECT @device directory SUBSTRING (filename, 1, CHARINDEX(N'master.mdf', LOWE 


(OG [EXECUTE (M'CREATE DATABASE Northwind 
| ON PRIMARY (NAHE = N''Northwind'', FILENAME = N''' 
LOG ON (NAME = N''Northwind_log'', 
go 


Dee 


H FROM master.dbo.sysaltfiles WHERE dbid = 1 AND fileid = 


+ @device_directory + N'nor 
FILENAME = N''' + @device_directory + N'nc 


1 


+ 
> 


(Sy Resuts | 23 Messages | 


Solution Explorer |f Script Explorer 


| value Type -| _| Name cen 


Y @device_director, C:\Program Files\Microsoft SQL Server\MSSQLQ, + nvarchar | (>) instnwnd.sqi{) Line 19 Trans 


_| Name 


a Y M 


(tocas Ewah |ECal Stack [= Breakpoints | 577 Command window |E Output 
Ready tn 19 Coll chi INS z 
Figure 4 


SQL Server 2008's T-SQL debugging functionality 


script variables and their values. You can change the 
variable values by clicking the value in the Value 
column of the Locals window and then altering the 
contents of the Value field. The running call stack is 
displayed in the Call Stack window, which is shown 
in Figure 4. 

You can use the Step Over and Step Into buttons 
on the debugging toolbar, or F10 and F11, to single 
step through your T-SQL code. The current line of 
code is always displayed with a yellow arrow. You can 
set breakpoints by clicking in the left margin of the 
editor. Breakpoints are indicated by a red ball icon 
in the left margin. If your T-SQL code is running in 
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TSQL DEBUGGING SYSTEM 
REQUIREMENTS 


There are a few system requirements that you need to meet to use SQL 
Server 2008’s T-SQL debugger. First, you need to be aware that T-SQL 
debugging has both client and server components, which means you can 
debug only T-SQL code that’s running on SQL Server 2008 systems. Next, 
SSMS must be running under a Windows account that’s a member of the 
sysadmin fixed server role. 

If you're debugging T-SQL on a remote SQL Server 2008 system, you 
need to make sure that TCP port 135 is open. In addition, if the domain 
policy requires IPsec for network communications, UDP port 4500 and UDP 
port 500 must also be available. 
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-alx debug mode, it will stop when 


QuickWatch 
the executing script reaches a 
Expression: Reevaluate breakpoint. However, running 
= . 
Add Watch your T-SQL code using the 
Value: Execute button or F5 will run 


[Name ` <= | Value ROR =] e = the script, but not in debug 
4 @device_directory C:\Program Files\Microsoft SQL Server\MSSC Q + nyar mode, and any breakpoints 


will be ignored and the code 


Figure 5 x| | will run as usual. 
The QuickWatch win- You can also inspect the 
dow displaying variables _ cose | He | values of variables by using 
added to watch list L 


the QuickWatch window. To 
display and potentially change a variable in the 
QuickWatch window, start your T-SQL code in 
debug mode, highlight the variable that you want 
to inspect, and select the Debug, QuickWatch 
option from the menu or press Ctrl+Alt+Q to 
display the QuickWatch window, which is shown 
in Figure 5. 

The QuickWatch window displays all of the 
variables that have been added to the watch list. 
Variable names are listed in the Names column, 
contents are shown in the Values column, and 
the data type is listed under the Type column. 
To change a variable's value, simply position the 
cursor over the value in the Value column and 
alter the value that's displayed. 


The Bug Stops Here 
SQL Server 2008’s T-SQL debugger is an invalu- 
able addition to every DBA and developer’s 
toolbox. Although using the new debugger is 
pretty straightforward, there are a couple of 
caveats to making it work remotely. You can 
refer to the sidebar “T-SQL Debugging System 
Requirements,” page 25, for more information 
about the prerequisites for using the debugger. If 
you spend a significant amount of time writing 
T-SQL scripts and stored procedures, the new 
development and debugging enhancements are 
reason enough to upgrade to SQL Server 2008. 
SQL 
Usually, it’s harder to pinpoint. enan ea 
Amazing what you can accomplish once 
you have the information you need. 


When the source of a database-driven application 
slowdown isn't immediately obvious, try a tool that 

can get you up to speed. One that pinpoints database 
bottlenecks and calculates application wait time at each 
step. Confio lets you unravel slowdowns at the database 
level with no installed agents. And solving problems 
where they exist costs a tenth of working around it by 
adding new server CPU’s. Now that’s a vision that can 
take you places. 


A smarter solution makes everyone 
look brilliant. 


Editor’s Note 


This article has been 
adapted from Michael 
Otey’s SQL Server 2008 
New Features (Osborne/ 
McGraw-Hill). 
—Megan Keller, associate editor 


Download your FREE trial of Confio Ignite” at www.confio.com/sqimag 
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Need to keep your databases 
synchronized? Use this solution. 


BAs often encounter change-tracking prob- 
D- when they try to keep multiple copies 

of a database synchronized across various 
environments. In addition to the standard develop- 
ment, test, quality assurance, and production versions, 
there can be copies of the database for implementa- 
tions by divisions, departments, and development 
teams. You need to be able to check your databases 
quickly and easily to see if they’re synchronized to 
support application changes. 

We've devised a solution that lets you see whether 
copies of a database are synchronized, even when 
there’s no direct access between the SQL Server 
systems containing those databases. The solution pro- 
duces a report that shows discrepancies between copies 
of a database, and you can create this report whenever 
a database changes or at regular intervals. 

The database synchronization solution is relatively 
simple to implement because it uses metadata that’s 
extracted by running T-SQL queries against data- 
bases. The metadata, extracted from system tables and 
system views in two or more databases, is stored in 
flat files that are transported to a central location and 
loaded into a composite table. The table is queried to 
group and compare the objects, names, and attributes. 
The comparison verifies that the data type, length, 
nullability, and primary key position are the same for 
the column definitions in all the databases. 

The results of the comparison are put in the data- 
base synchronization report, which shows the columns 
and their attributes for each table, view, and index 
in each database. With just a glance at the report’s 
Summary field, you can find out whether there are 
discrepancies between the databases’ contents. 

To implement this database synchronization solu- 
tion, you need to perform the following steps: 

1. Create the composite table in a central location. 
2. Prepare the queries to extract the databases’ 
metadata. 
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3. Run the queries and transport the results. 
4. Load the results into the composite table. 
5. Run the database synchronization report. 


We’ll walk you through each step, then show you what 
to look for in the report. 


Step I: Create the Composite 
Table 

The process to obtain a database synchronization 
report starts with creating the composite table, which 
is named DB_META, in a central location. This table 
will hold the metadata extracted from each database. 
Web Listing 1 (www.sqlmag.com, InstantDoc ID 
101901) contains the code you can use to create it. 


Fred Schuff 


(fschuff@ system-support.com) is a consultant, 
currently designing databases and web-based 

applications that support the management 

of large, multi-site, data centers. Fred has an 

extensive background with database systems 


and various online environments. 


Don Ritchie 


is a senior technical analyst at Implementation 


Step 2: Prepare the Queries 

As mentioned previously, a T-SQL query is used to 
extract the metadata from each database. You can 
find this query in the MetadataExtractionQuery 
.sql file, which you can download by going to www 
.sqlmag.com, entering 101901 in the InstantDoc ID 
text box, and clicking the 101901.zip hotlink. This 
query extracts metadata from columns in a database’s 
tables, views, and indexes. A UNION opera- 
tion is used to combine these data sets, and 
the metadata is inserted into a table named 
ZDB_META. 

Table 1, page 28, shows the 10 fields in 
the ZDB_META table. Note the first field named DB_ 
Identifier. Each database you want to include in the 
database synchronization report must have a database 
identifier, which is a one-character value that uniquely 
identifies that database. You can use uppercase and 
lowercase letters and single digits, so you can include 
up to 62 databases in the report without using special 
characters. The one-character values allow the results 
for each column to be aligned in the report’s Summary 
field, making it easy to compare those results. 


and Consulting Services, which is headquartered 
in Newtown Square, Pennsylvania. 


ORE on the WEB 


Download the code at 
InstantDoc ID 101901. 
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TABLE |: The 10 Fields in the DB_META Table 


Field 
DB_Identifier 
Object 

Object Type 
Target 

Target Type 
Column name 
data type 
Max-length 
NULLable 
Key Position 
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Field’s Data Type Field Description 


CHAR(1) User-defined unique identifier (e.g., 1, 2, 3, A, B, a, b) for the database 
NVARCHAR(128) Name of the object 

CHAR(12) Type of object (i.e., table, view, or index) 

NVARCHAR(128) Name of the parent object 

CHAR(12) Type of the parent object 


NVARCHAR(128) Name of the column in the table, view, or index 
NVARCHAR(128) Data type of the column 


Web Listing 2 shows how to set up 
a batch file that can be scheduled to 
run as a background process. The 
first command in Web Listing 2 
uses the sqlcmd.exe utility to open a 
command window and run the cus- 
tomized query. (Alternatively, you 
could use osql.exe or an equivalent 
SQL Server utility to run the query.) 
In this command, you use the 
-U and -P options to specify your 
login credentials and the -d and -S 


INT Maximum length of char type fields (fixed items such as INT = 0) l . 

, options to specify the database you 
CHAR(3) Column is NULLable (YES) or not NULLable (NO) want to run the query against and 
INT Column position in key or 0 (in table data, this is the primary key) the server on which that database 


You need a customized query file for each database 
you want to include in the database synchronization 
report. Creating a customized query file is simple. 
First, make a copy of MetadataExtractionQuery.sql 
and rename it so that the filename includes the data- 
base identifier. For example, if you intend to run the 
query file against the database that has the identifier 
of 1, you can name the file something like Metadata 
ExtractionQuery1.sql. 

After you’ve renamed the query file, you need to 
open and modify it. In two spots, you'll find the code 


"database-number-assigned' AS [DB_Identifier], 


In this code, replace database-number-assigned with 
the database identifier. 
In two spots, you'll find the code 


isc.table_catalog = 'database-name' 


In this code, replace database-name with the database’s 
name. 

If you need to specify a specific schema to identify 
the database columns, you must make an additional 
modification. You need to uncomment the line 


-- AND isc.table_schema = 'schema-name' 


and replace schema-name with your schema’s name. 
MetadataExtractionQuery.sql’s comments indicate 
where to make the four required and one optional 
modifications. 


Step 3: Run the Queries and 
Transport the Results 

At this point, you’re ready to run the customized 
queries against the databases. For each database, you 
can create a batch file that not only runs its custom- 
ized query but also transports the query’s results to a 
flat file on an FTP server. (The data from individual 
databases can be extracted and collected in a variety 
of ways. We chose a simple process of extracting, 
transporting, and reloading the data.) 


resides. You use the -i option to 
specify the customized query file’s pathname (e.g., 
\\SQL3\DBSyncReport\MetadataExtractionQuery 1 
.sql). The -o option indicates the pathname of the 
output file that will contain the query’s results (e.g., \\ 
SQL3\DBSynchReport\MetadataExtractionQuery1 
.txt). The -h and -w options specify the header and 
width information. You don’t need to change these 
values. 

The second command in Web Listing 2 uses the 
bep.exe utility to transfer the query’s results to a flat 
file. In this command, you specify the table you want 
to export (which will always be DB_META), the 
database in which that table resides, and the database’s 
owner in the format //database. Jowner. ]DB_META. 
You use the -o argument to specify the name of the 
flat file that will receive the exported data (e.g.,\\SQL3\ 
DBSynchReport\MetadataExtractionData1.txt). You 
use the -S argument to identify the SQL Server 
instance containing the DB_META table and the -U 
and -P arguments to provide your login credentials. 
The -c argument indicates that the operation needs to 
be performed using a character data type. 

The last command in Web Listing 2 uses the FTP 
command to copy the flat file to an FTP server. Spe- 
cifically, the command uses the -s parameter to specify 
a text file that contains the commands to run. For 
example, you might specify \\SQL3\DBSynchReport\ 
MetadataExtractionFTPCommand1.txt as the text 
file and include the following commands in it: 


OPEN servername-or-IP-address 
userid 

password 

PUT outfile-name 

BYE 


where servername-or-IP-address is the name or IP 
address of the FTP server you want to access using 
the specified credentials (userid and password) and 
outfile-name specifies the name of the flat file you 
want to copy to the FTP server. This filename should 
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be the same one you used with the -o 
argument in the BCP command. Note 
that the exact commands in the text file 
will depend on your FTP version. Check 


Table 
Table 


ore og š Table 

your versions documentation. Table 
Table 

Step 4: Load the Table 
Results into the Table 


Composite Table 

The next step is to copy each flat file 
from the FTP server to the central loca- 
tion (i.e., the database that contains the 
composite DB_META table), then load 
its contents into the composite table. 
Once again, you can use a batch file to 
automate these tasks, as Web Listing 
3 shows. The load process executes a 
T-SQL command to delete existing rows 
in the composite DB_META table and 
then executes the BCP SQL utility to 


Table 
Table 
Table 
Table 
View 
View 
View 
View 
View 
View 


Object-Type 


Table-Index 
Table-Index 
Table-Index 
Table-Index 


Object 
Tablet 


Parent 
Table1 


Table1 = Tablet 24 
Table1 = Tablet Column2 INT 

Table1 Tablet Column3 FLOAT 

Tablet Tablet Emma CHAR 30 
Tablel Tablei ma CHAR 32 
Table1 Tablet = Columns CHAR 12 


T2-IX1 of 
T2-IX1 of 
T2-IX1 of 
T2-IX1 of 
Table2 
Table2 
Table2 
Table2 
View1 
View! 


Table1 
Table1 
Tablet 
Tablet 
Table2 
Table2 
Table2 
Table2 
View! 
View1 
View1 
View1 
View1 
View1 


Column01 
Column02 
Column03 


Column11 
Column12 
Column13 


Column31 
Column32 
Column33 
Column34 
Column35 


View! 
View! 
View1 
View! 


Data-Type Max-Length Key-Pos 


e 


load the new data records into the com- 
posite DB_META table. The scripts are very similar 
and reference a SQL command file and the BCP data 
file as input (-i) rather than output (-o). 

Using batch files to run the queries, transport the 
queries’ results, and load the results into the composite 
table eliminates the need for a last-minute rush to ini- 
tially collect the metadata. You can also use batch files 
for the ongoing collection of metadata. The batch files 
to run the queries and transport the results would be 
similar on each server, as would the batch files to load 
the data into the composite table, but some details 
would need to change. 

When there’s a new flat file that contains updated 
metadata for a database, you can run a batch file that 
copies the new flat file from the FTP server to the 
central location, deletes all the existing rows in the 
composite table for that database, then runs bep.exe 
to append the new flat file’s contents to the composite 
table. This batch script file might look like the one in 
Web Listing 4. 


Step 5: Run the Report 
After the composite table contains the metadata for all 
the databases, you can run DatabaseSynchronization- 
Report.sql, which you'll find in the 101901.zip file, to 
create the report. DatabaseSynchronizationReport.sql 
is written to compare five databases. To compare more 
than five databases, make three modifications. 
Suppose you want to compare six databases. To do 
so, you first need to change the Summary column code 
in Web Listing 5. Change the code in callout A to 


",'+RTRIM(ISNULL (DB5.DM_DBID, '-'))+ 


' „ '+RTRIM(ISNULL (DB6.DM_DBID, '-'))+ 
AS [Summary] 
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Next, you need to add an additional LEFT OUTER 
JOIN statement, which Web Listing 6 shows. You'd put 
this statement after the last LEFT OUTER JOIN state- 
ment in DatabaseSynchronizationReport.sql. 

Finally, you need to change the GROUP BY code 
in Web Listing 7. Specifically, you need to change the 
code in callout A to 


","+RTRIM(ISNULL (DB5 .DM_DBID, '-'))+ 
",'+RTRIM(ISNULL (DB6.DM_DBID, '-')) 


Understanding the Report 
The database synchronization report presents the con- 
tents of all the queried databases so that the informa- 
tion can be viewed together. The data for each object 
is grouped and sorted by object and column name. 
Figure 1 shows a sample report for three queried 
databases. 

The report’s Summary field shows whether there’s 
a discrepancy across the databases. The Summary field 
of each column record should list the database identi- 
fiers for all the queried databases (e.g., “1.2.3.4.5” for 
five queried databases, “1.2.3.-.-” for three queried 
databases). This makes missing columns easy to spot. 
For example, in the sample report in Figure 1, we 
highlighted a missing-column discrepancy in yellow. 

Attribute discrepancies appear as multiple entries 
for a column. In Figure 1, we highlighted this type of 
discrepancy in purple. The information in the Data- 
Type, Max-Length, Key-Pos, and Null fields can help 
identify what attribute differs between the databases. 

Note that the report includes an object-separator 
row at the start of each new object. These rows split 
the results into groupings, making it easier to find and 
differentiate between objects. 


Figure | 


Annotated sample 
database 
synchronization 
report 
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Adapt as Needed 
This solution is highly adaptable. For example, we 
designed MetadataExtractionQuery.sql and Database- 
SynchronizationReport.sql to work with metadata in 
SQL Server 2008, 2005, and 2000 databases, but you 
can adapt these scripts to work with similar data in 
other database systems, such as Oracle or DB2. 
Similarly, we designed the solution to work in an 
environment in which multiple copies of a database 
reside on servers that can't directly access each other. 
However, this solution will work in all types of environ- 
ments. If there's network access between servers or if 
multiple database copies (with different names or in 
different instances) reside on the same server, you can 
streamline the process. You might be able to load the 
metadata in the ZDB_META tables directly into the 
DB_META table. Your environment will determine the 
best extraction, transport, and load methods to use. 
You can even adapt the database synchronization 
report to better meet your needs. For example, you can 
remove the header row or the object-separator rows. 
Or, if all you want to see is the Summary field for each 
column record, you can add a WHERE clause to the 
report query to select only those columns. If there are 


Pri 


five queried databases, you'd add the clause 


WHERE RTRIM(ISNULL(DB1.DM_DBID, '-'))+ 
' „. '+RTRIM(ISNULL (DB2.DM_DBID, '-'))+ 
' „. '+RTRIM(ISNULL (DB3.DM_DBID, '-'))+ 
' . '+RTRIM(ISNULL (DB4.DM_DBID, '-'))+ 
' . '+RTRIM(ISNULL (DB5.DM_DBID, '-')) 
<> *T.2. 344.5" 


Find and Fix Discrepancies 
Before They Cause Problems 
Discrepancies between copies of a database might not 
seem like a significant problem, but the impact of a 
missing database change can be quite serious. Errors 
can delay releases, even for weeks, hurting your sched- 
ules and your project's credibility. 

Manually rechecking column definitions in large 
numbers of tables, views, and indexes is tedious and 
error-prone, so try the database synchronization 
solution. Using the information from the database 
synchronization report, you'll be able to quickly find 
and fix all discrepancies so that your database copies 
are perfectly synchronized. SQL] 
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Rebuil 
in 


Online 


Customize this online index job for your 
SOL Server environment 


ecently, I was tasked with maintaining 
R indexes under a five nines model. I was told 

by my boss that both the blocking caused 
by reorganizing indexes and the table locking caused 
by offline index rebuilds are unacceptable. Because 
our SQL Server environment is made up of all SQL 
Server 2008 and 2005 Enterprise Edition systems, I 
decided to avoid these problems by performing online 
index rebuilds. I set out looking for an appropriate 
customizable solution. Ultimately, I wound up cre- 
ating my own solution, an online index job, which I'll 
share in this article in hopes you find it as useful in 
your environment as it is in mine. I'll show you how 
I created the job, how to implement the job, and the 
various job parameters. 


Creating the Online Index Job 
My starting requirements for the job were simple. 
I first needed to identify which indexes could be 
rebuilt online. A quick look at MSDN revealed cer- 
tain guidelines. Large object (LOB) data types (e.g., 
image, ntext, text, varchar(max), nvarchar(max), 
varbinary(max), XML) must be created, rebuilt, or 
dropped offline. Nonunique, nonclustered indexes can 
be created online when the table contains LOB data 
types, but none of these columns are used in the index 
definition as either key or nonkey (included) columns. 
Nonclustered indexes defined with LOB data type col- 
umns must be created or rebuilt offline. It’s important 
to identify which indexes can be rebuilt online because 
issuing a rebuild with the online index option on an 
index that fits the above offline criteria will result in 
an error, leaving your index unprocessed. 

After identifying which indexes could be properly 
rebuilt online, I added various parameters to dif- 
ferent steps of the index job. For indexes that can’t 
be rebuilt online, you can choose to reorganize them, 
ignore them, or rebuild them offline (if your business 
model allows). I also added parameters that let you 
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reorganize indexes if they’re under a certain fragmen- 
tation percentage and rebuild them if they’re over 
a certain fragmentation percentage. Then I built in 
features to track each index’s duration, start time and 
end time, progress, fragmentation before and after 
running the job, and any resulting blocking. Lastly, 
I included an option to exclude databases and tables 
from index operations. 

But didn’t my boss say blocking caused by reor- 
ganizing was unacceptable? To address this problem, 
I tracked the SPID of the index operation and cre- 
ated a sub-job to monitor any blocking that this 
operation might cause. If blocking exists for more 
than x minutes in a row and blocks more than x 
processes, the SPID is killed, a period of time passes, 
and the index operation is retried. 

According to Microsoft, reorganizes 
don’t hold long-term blocking locks; there- 
fore, SQL Server won't block running 
queries or updates. However, in high-trans- 
action environments that can exceed 1,800 transac- 
tions per second, as in my environment, you have 
to be very careful with reorganizations, hence my 
blocking detector. 

So what happens if the blocking detector kills a 
SPID that’s in the process of performing a rebuild? 
Because rebuild operations occur as single transac- 
tions, the rollback incurred from killing an offline 
rebuild on a large index that’s near completion would 
be far worse than simply letting the operation continue. 
For this reason, I don’t target them with the blocking 
detector. I also don’t target online rebuilds for various 
reasons, such as possible performance implications 
related to reverting mapping indexes. According to 
“Microsoft SQL Server 2005 Online Index Opera- 
tions” (technet.microsoft.com/en-us/library/cc966402 
.aspx), “There are only very short periods of time in 
which concurrent DML and select operations are pre- 
vented.” Any blocking should be tolerable given the 
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alternatives. A reorganize is a different story. Because 
a reorganize acts only on a pair of pages at a time, if 
it's killed, only the current swap operation is affected. 
The next time a reorganize is issued, the job picks up 
where it left off. 


Implementing the Index Job 

To create the index job, run the code, which you 
can download at www.sqlmag.com, InstantDoc ID 
101777, in the following order: 

1. usp_update_schedule_enabler.sql 

2. Index_BuildIndexKey.sql 

3. Index_ProcessIndexes.sql 

4. Index_PostCleanup.sql 

5. Index_PostFragCalc.sql 

6. Index_HandleBlocking.sql 

7. job_Blocking_Killer.sql 

8. job_Indexing_Master.sql 


Note I use a database called “Common” to house 
common functionality amongst multiple servers. 
Before you run these executable files, you'll 
need to replace Common in each script with the 
references to suit your environment. Also, in the 
Index_BuildIndexKey procedure I create tables 
on a “Tables” file group. You'll need to change 
Tables to the name of your file group or the job 
will fail. 

When you run job_Indexing_ Master.sql, you'll get 
four non-existent step warnings. They’re safe to ignore. 


Select apage || K S Script ~ [3 Help 
> General 
LA Schedules 
> Alerts 
> Notifications 1 
H Targets 2 Enable 5 minute log backups 
3 Enable Blocking Killer 
4 Switch BULKLOGGED 
5 Index Job - The Brains 
6 Log Errors 
7 Disable Blocking Killer 
8 Disable 5 minute log backups 
9 Switch Recovery FULL 
10 Calculate Fragmentation Differences 


Figure Ï! Each step in a job has to be created sequentially, and 

The DB Maint — Index some steps are set up to skip to future steps in the 
Maint Job step list event of a step failure. For example, if Step 2 fails, the 

job goes to Step 8. Well when Step 2 is created, Step 

8 obviously doesn't exist yet, and a warning is issued. 

After running the provided scripts, you'll 

see two new disabled SQL Server Agent jobs: 
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DB Maint - Blocking Killer and DB Maint — Index 
Maint. I named the steps of the DB Maint — Index 
Maint job to be pretty self-explanatory. The following 
is a quick overview of the steps: 

* Step | builds a tracking table that’s consumed 

by all subsequent steps to perform their 
operations. This step contains both an edition 
check and version check. If you aren’t using the 
appropriate edition and version for online rebuilds, 
all of the indexes targeted for rebuild will be done 
so offline. 

Step 2 and Step 8 are put in place to adjust the 
duration of your log backups if necessary. This 
functionality helps keep your transaction logs 
from growing too large and filling depending on 
your autogrowth and drive configurations. Keep in 
mind that your transaction logs won't clear during 
a rebuild operation because, as I stated earlier, a 
rebuild operation occurs as a single transaction. 
Step 3 and Step 7 enable and disable the 
previously created DB Maint - Blocking Killer job 
to monitor the amount of blocking caused by a 
particular index operation. If you don’t want this 
feature to run, simply delete these steps. 

Step 4 and Step 9 are in place if you can switch 
your recovery model in your environment. I have 
included the Microsoft best practice method in 
each job step regarding switching recovery models 
on the fly. I included this feature because a rebuild 
is a bulk logged operation, meaning it’s minimally 
logged when your database is in the bulk-logged 
recovery model. According to “SQL Server 2005 
Online Index Operations,” when rebuilding an 
index in a bulk-logged database, the log-to-data 
ratio is roughly 10 percent of the full recovery 
log-to-data ratio. Minimal logging is helpful when 
you want to keep transaction log sizes manageable, 
especially in replicated environments. 

Step 5 processes all the indexes in the table created 
by Step 1. You can change the retry interval and 
retry attempts on this step to the desired minutes 
to wait in the event the blocking killer terminates 
this step. 

Step 6 logs any failed or incomplete operations to 
the SQL Server error log. 

Step 10 goes back and recalculates fragmentation 
so that you can see a before and after snapshot. 
Step 10 and Step 1 use the sys.dm_db_index_ 
physical_stats dynamic management view (DMV) 
to gather information about indexes, which 
requires only an intent-shared (IS) table lock, 
regardless of the mode that it runs in. 


Figure | shows all of the above steps listed in SQL 
Server Management Studio (SSMS). 
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Job Parameters 

You need to understand how all of various 
input parameters are utilized to custom 
tailor the index job to your environ- 
ment. The stored procedure in the first 
step of the DB Maint - Index Maint 
job (Build Indexing Key) accepts a few 
parameters. @minFragPercent is the min- 
imum percentage of fragmentation an 
index must possess before it’s considered 
for an index operation. You can set the 
minimum percentage of fragmentation 
based on your specific needs. Anything 
below this threshold will be ignored. 
@maxAttempts are the number of times an 
index operation that was previously killed 
by the DB Maint - Blocking Killer job for 
causing excessive blocking will be retried. 


IES Job Step Properties - Index Job - The Brains 
Selectapage = 7 N Script v G Help 


REBUILD INDEXES ONLINE be n =) 


Step name: 
[Index Job - The Brains 


Type: 


[Transact SQL script (T-SQL) 


EXEC Common.dbo.Index_ProcessIndexes 
Command: @reorgMinFragPercent = 20, 
annarra REN = 20, 
Open... meUniy = Ï, 
Oren. | @reorgNonOnlines = 0, 


Select All | @globalAllowReorgs = 0 


If you want to target only one or two 
databases, use @databaseIncludeList, which is a 
comma delimited varchar. If you want to exclude only 
a few databases, use @databaseExcludeList, which is 
also a comma delimited varchar. Lastly, if you want to 
exclude only a particular table, you can do so by using 
@tableExcludeList. 

The stored procedure in Step 5 of the index job 
also accepts a few parameters, which Figure 2 shows. 
@reorgMinFragPercent is the minimum amount of 
fragmentation that must be present for a reorgani- 
zation to occur. This value shouldn’t be below the 
@minFragPercent value in the Build Index Key step. 
@rebuildMinFragPercent is the minimum fragmenta- 
tion that must be present for a rebuild to occur. For 
example, if you set these values to 20 percent and 
30 percent, respectively, anything below 20 percent 
fragmentation will be ignored, anything between 20 
and 30 percent fragmentation will be reorganized, and 
anything greater than or equal to 30 percent will be 
rebuilt online if possible. 

If you set the two percentages to equal each other, 
only rebuilds will occur. @onlineOnly set to 1 means 
you're going to perform only online rebuilds. Setting 
this value to 0 will result in indexes being built online 
where possible and offline where it’s not possible. 
@reorgNonOnlines when set to | in conjunction with 
@onlineOnly set to 1 lets indexes that can’t be rebuilt 
online be reorganized. I added @globalAllowReorgs 
to be a final fail safe that lets reorganizations occur 
on a global scale. In my environment it’s absolutely 
critical that some indexes not be reorganized. If all 
of your indexes can be reorganized, @globalAllow 
Reorgs can be set to 1. 

Lastly, the DB Maint - Blocking Killer contains 
one step that includes a stored procedure that accepts 
two parameters. @blockingMins is the number of 
minutes a SPID must be detected as blocking. This 
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job runs every minute and performs the blocking Figure 2 
check. If it finds the SPID performing an index The Index Job - The 
operation blocking @blockingMins in a row, it will Brains parameters 
kill the SPID. To define blocking I added the @ 
blockedProcesses parameter. If the blocking SPID is 
blocking more than @blockedProcesses processes, it’s 
considered to be an offender and its blocking count is 
iterated. 
For the sake of time 


and space in this article, Because rebuild 
1 have included exte Operations occur as single 


sive documentation in 


the code describing what transactions, the rollback 


each step performs and 


the parameters they incurred from killing an 
take. 


Once you've config- offline rebuild on a large 
ured DB Maint - Index . 9 
index that's near 


Maint to your liking, 


including "specifying completion would be far 


a schedule, be sure to 

enable it in SSMS by worse than simply letting 
right-clicking the job f Z 

and selecting Enable. the operation continue. 


Maximize Index Performance 
This solution lets you get maximum performance out 
of your indexes without having to sacrifice uptime. I 
could have added functionality to this job seemingly 
forever, but I realized no job is written perfectly or 
applied to all applications. In the code, you'll notice 
a few comments hinting at possible expansion. These 
are a few extra parameters that can be utilized, 
expanded upon, or deleted to tailor this job to your 
environment. I hope this index job helps you easily 
perform online index rebuilds. [SQL 
InstantDoc ID 101777 
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SQL Server Backup and 
Recovery Software 


Learn when and how to upgrade to a third-party solution 


D of SQL Server backup and recovery solu- 
tions exist, ranging from SQL Server's own native 
backup functionality to offerings sporting rich feature- 
sets (and hefty price tags). I’ve collected tips and advice 
from SQL Server backup and recovery experts to help 
you decide when you might need to upgrade from SQL 
Server’s native backup capability to a more robust 
third-party solution, and what features to look for 
when you decide to upgrade. 


Back(up) to the Basics 

When you begin looking for a SQL Server backup and 
recovery solution, your first stop should be with SQL 
Server itself: If you're dealing with a relatively small 
database, SQL Server 2008/2005’s native backup and 
restore features might be sufficient. “I typically tell cus- 
tomers that SQL Server’s native backup functionality 
works very well, and is a very viable solution in most 
environments,” says SQL Server Magazine contrib- 
uting editor (and SQLServerAudits.com consultant) 
Michael K. Campbell. 

Although native backup and restore might fit the 
bill for some database projects, Brent Ozar—Quest 
Software’s SQL Server domain expert—points out the 
deficiencies of SQL Server 2005 native backups in a 
post on his personal blog (tinyurl.com/chxc5n). Ozar 
explains that native SQL Server 2005 backups lack 
data compression, so backup files tend to be massive. 
Writing backup files can be terminally slow, and Ozar 
points out that “SQL Management Studio doesn’t 
come with reports about the backup process.” SQL 
Server 2008 introduces data compression and encryp- 
tion into the native backup process, but SQL Server 
DBAs with massive databases to manage might want 
to look at more comprehensive backup offerings. 


When to Upgrade to a Backup 
and Recovery Product 

How do you know it’s time to upgrade to a more 
comprehensive backup solution? Campbell says he 
considers a number of use-cases that would lead him to 
recommend third-party solutions to clients. “[If] they’re 
running low on disk because [they're] keeping 2 to 3 
days’ worth—or more—of backups, which takes up too 
much premium disk space, or if their data is sensitive 
enough that it should be encrypted while at rest.” Other 
situations in which Campbell recommends the use of a 
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more robust backup solution include if you need to use 
your network to keep offsite copies of backups and your 
databases are very large or you could otherwise benefit 
from a third-party product’s 40 to 60 percent compres- 
sion. Finally, Campbell also leans toward third-party 
solutions if you're dealing with extremely large databases 
that need to be backed up on a regular basis. 


Features to Look For 

What features are important in a SQL Server backup 
and recovery product? “[I would look for] the ability to 
do online backup, the ability to restore a table or file 
group, and the ability to back up full text catalogs,” says 
Michael Otey, technical director for SOL Server Maga- 
zine. “Vd also look for the ability to back up FileStream 
objects, [as well as data] compression, encryption, and 
a T-SQL command-line interface.” 

Campbell has reservations about choosing a solu- 
tion that requires installing too much software into 
an IT environment. “I’m a bit leery of the kinds of 
software third-party vendors deploy on machines in 
order to enable new features and functionality,” explains 
Campbell. “Ideally, configuring a target server for third- 
party backup should just consist of dropping a couple 
new special stored procedures in the master database, 
and registering a couple of .dlls on the box.” 


The Importance of Backup 
Scheduling 
“Remember that backups that aren’t regularly checked 
for viability are potentially worthless,” says Campbell. 
“I always tell my clients: "We've all worked in compa- 
nies where a server or something critical has crashed, 
and the folks in IT looked like complete idiots because 
their backups weren’t working correctly.’ Likewise, I 
always mention that CEOs and other executives usu- 
ally don’t know a thing about backup software, other 
than it’s supposed to protect them if something goes 
wrong, so failure to regularly tests backups and work 
through simulated recoveries is a capital offense. In 
other words, the worst time to learn how to recover a 
database—either from native SQL Server backups or 
with third-party software involved—is after the sales 
database has crashed and while 300 people are waiting 
for it to be recovered before the business can become 
profitable again.” 
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Backup SQL Server Server 
$995 per server Contact $99 for 1 $799 per server $1,195 per Contact vendor $295 Lite 
vendor license; $89 for instance Edition; $795 
2-5 licenses; Pro Edition 
$79 for 6-10 
licenses 
SQL Server SQL Server 2008, | SQL Server SQL Server SQL Server SQL Server 2008, | SQLServer 2008, — SQL Server 
Versions 2005, 2000 2008, 2005, 2008, 2005, 2008, 2005, 2005, 2000; SQL | 2005, 2000; SQL 2008, 2005 
Supported? 2000; SQL 2000; SQL 2000 Server 7.0 Server 7.0 
Server 7.0 Server 7.0 
Number of Unlimited Unlimited Unlimited Unlimited Unlimited Unlimited Unlimited 
Supported 
SQL Server 
Instances? 
Backup Local disk, USB Local disk, Local disk, USB Local disk, USB Local disk, USB Local disk, USB Local disk, 
Devices? storage, iSCSI, USB storage, — storage, iSCSI, storage, iSCSI, — storage, iSCSI, storage, iSCSI, FC USB storage, 
FC SAN, NAS, IP iSCSI, FC FC SAN, NAS, FC SAN, NAS, IP FC SAN, NAS, IP SAN, NAS, IP SAN, — iSCSI, FC SAN, 
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to-Tape (D2D2T) 
Backups? 
Back Up Data Data at-rest Both Both Both Both Data at-rest Both 
At-Rest? Data 
In-Flight? Both? 
Data Yes Yes Yes Yes Yes Yes Yes 
Compression? 
Data No Yes Yes No No No No 
Deduplication? 
Point-in-Time Both Both Both Point-in-time Both Both Both 
Recovery? 
Point-of-Failure 
Recovery? 
Both? 
Restore Tables? [MW Both No Both Tables Both No 
Individual 
Objects? Both? 
Bare-Metal Yes Yes No No No No No 


Restore? 
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Brian Goldfarb on ASP.NET 
AJAX, MVC, and Silverlight 

uring a recent visit to Microsoft, the 

Windows IT Pro staff got to sit down with 
Brian Goldfarb, director of Microsoft’s development 
platform group. Brian pointed out that a big focus for 
Microsoft development tools has been in the area of 
Rich Internet Applications (RIA). Brian shared with 
us some of Microsoft’s newest web developments, 
including Microsoft’s ASPNET AJAX, the new ASP. 
NET Model-View-Controller (MVC) framework, and 


Silverlight. 
ASP.NET AJAX 
allows developers to 


create web applications 
that include rich and 
responsive UI compo- 
nents such as tooltips, 
pop-up windows, and 
progress indicators. ASP 
.NET AJAX is compatible 
with the most frequently 


Internet Explorer, Firefox, 
and Safari. The goal is not just to provide cool-looking 
UIs, but to also enable applications to provide a great 
platform for business applications that make business 
processes work better. 

Brian noted that the new ASP.NET MVC frame- 
work is a response to Web 2.0. ASP.NET MVC 
separates web application components into models 
that implement application logic, views that imple- 
ment the application’s UI, and controllers that handle 
user interaction and select the view to show to 
the user. MVC applications are supported via 
new project templates that are part of Visual 
Studio 2008. The new ASP.NET MVC project 
support makes it easier to handle application 
complexity and perform testing. Brian pointed 
out that it’s important to understand that ASP 
.NET MVC is an alternative to traditional 
ASP.NET web forms and that Microsoft fully 
intends to continue support for both ASP 
.NET MVC and ASP.NET web forms going 
forward. 

Brian also mentioned that Silverlight has 
been rapidly gaining in adoption and that 
there have been approximately 300,000,000 
Silverlight downloads so far. Brian showed us 
an amazing demo of the Hard Rock Café’s new 
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Microsoft Silverlight has 
been rapidly gaining in 
adoption and there have 
been approximately 
300,000,000 Silverlight 


used browsers, including downloads so far. 


Silverlight implementation. This demonstration high- 
lighted Silverlight’s Deep Zoom technology. It started 
with a view of Paul McCartney’s signature. Then the 
demo began the process of zooming out. The next view 
showed the signature was actually an inscription on 
a figurine. Then the view zoomed out again to show 
the figures were a part of Hard Rock Café storefront 
display. Each time the view zoomed out you can see a 
larger view than the last. The next zoom showed that 
the storefront display was actually one tile in a 4 x 4 
tiled postage stamp. Finally, the view ended up showing 
that the stamp was part of letter mailed by Paul Mc 
Cartney. The resolution 
at each level was crystal 
clear and was legitimately 
cool. You can check out 
the Hard Rock Café’s Sil- 
verlight implementation 
at www.hardrock.com on 
the Memorabilia link. To 
see the current postage 
stamp view (which is a bit 
different from the one we 
saw demoed), select The 
Beatles link and then begin clicking the letter shown in 
the bottom row third image from the left. (See an image 
of the letter below.) 

Another really well-known site that Brian men- 
tioned that was built using Microsoft technologies is 
MySpace. The site (www.myspace.com) is built with 
100 percent Microsoft technology, including .NET 
Framework, ASP.NET, IIS, and SQL Server. SQL 
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Studio Keyboard Shortcuts 


sing shortcuts in SQL Server Management 

Studio (SSMS) can help you be more productive. 
Here’s a collection of my most frequently used SSMS 
keyboard shortcuts. 


Connect to Server: ALT+F, E. Connect to and 
manage the SQL Server relational database engine 
as well as Analysis Services, Integration Services, 
Reporting Services, and any local instances of SQL 
Server Compact Edition. You'll be prompted for server 
type, authentication. Object Explorer displays the con- 
nected server. 


Display Object Explorer: F8. In Object Explorer, 
where you perform most management functions, a 
hierarchical tree view displays all of the connected 
SQL Server server types. Right-click any object for a 
context-sensitive menu that lets you perform actions for 
the selected node. 


Scroll Up, Down: UP Arrow, DOWN Arrow. When 
the SSMS Object Explorer window is displayed, scroll 
up and down in the tree hierarchy by using the up and 
down arrows on the keyboard. 


Expand, Collapse Tree Nodes: +, -. The plus sign on 
the numeric keypad expands an Object Explorer tree 
node, and the minus sign collapses it. The right and left 
arrows also expand and collapse nodes, respectively. 


Object Explorer Details: F7. Display the details 
of each different node in the Object Explorer tree. 
The window is automatically updated as you navigate 
through the different database objects. 


Display Registered Servers: CTRL+ALT+G. This 
shows SQL Server Local Server groups and Central 
Management Server groups. Local Server Groups are 
related SQL Server systems that SSMS connects to. 
Central Management Server groups store groups of 
SQL Server systems that let you perform management 
tasks to be enacted on all servers in a group. 


Display Template Explorer: CTRL+ALT+T. Use 
this shortcut to see SSMS’s built-in T-SQL templates. 
These T-SQL scripts let you perform common manage- 
ment tasks such as backups and creating and dropping 
SQL Server databases and objects. If you want to add 
your own custom templates, they'll also be listed in the 
Templates window. Object Explorer’s shortcuts let you 
navigate this area. 


Display the Browser Window: CTRL+ALT+R. 
SSMS built-in browser is a little-known, fully capable 
web browser that runs within SSMS and uses Win- 
dows Live as a search engine. By default it connects 
to the SQL Server 2008 website: www.microsoft.com/ 
sqlserver/2008/en/us/default.aspx. [SQL 

InstantDoc ID 101868 


In the highly anticipated voting results for the 
February 2009 Instant Poll “What is your 
favorite type of SQL Server Magazine article?” 
(sqlmag.com/go/ArticleInstantPoll), 52 percent of par- 
ticipating readers voted in favor of step-by-step tech- 
nical articles. Although | saw tons of golden statues 
handed out at the Academy Awards that month, | 
have yet to see any trophies go our authors’ way. 

So to honor our great technical content, |’m fol- 
lowing up with an award of my own: Best Step-by- 
Step Column. And the nominees are: 

T-SQL Black Belt, Itzik Ben-Gan’s ever-popular 
collection of T-SQL tips and tricks, offers a boost in 
skills for advanced SQL Server users. His “Handling 


Arrays as Inputs” (InstantDoc ID 100657), shows 
how to use a T-SQL split function to handle arrays as 
inputs for SQL Server stored procedures. 

Solutions by Design, Michelle A. Poolet’s long- 
running series, most recently focused on data 
warehousing, offers helpful guidance. See her sub- 
scriber-exclusive article “Seven Steps for Successful 
Data Warehouse Projects” (InstantDoc ID 101562). 

In a Nutshell, Kevin Kline’s online blog, provides 
you with satisfyingly quick how-tos. | like his “Inter- 
preting Results from SQLIOSIM” (InstantDoc ID 
101048), which shows how to make something of 
the results that you get from the SQLIOSIM tool. 


Christan 
Humphries 


(christan.humphries @ penton.com) is a 
Windows IT Pro associate and regular 
contributor to Windows IT Pro, SQL 
Server Magazine, and associated websites. 
She specializes in the Windows IT Pro 
network of tools and resources. 
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see exactly which tables or indexes are being contended for without any manual effort. cs a (0) 


Generate alerts with full block chain details whenever a block occurs, when a block 
exceeds a specific duration, or when a blocking query contain specific text. 


Graphical display of all deadlock types, synchronized with an innovative grid-based 
display to provide the most complete picture of deadlocks available. 


Deadlock nodes contain the concise information you need to quickly visualize the 
deadlock and determine the root cause. 


Generate alerts including the deadlock victim SQL whenever a deadlock occurs, or 
when the victim query contains specific text. SQL Server Deadlocks 


Performance Dashboard with relevant SQL Server = + Top SQL analysis highlights heaviest queries 


elytee manie e Graphical blocking and deadlock analysis 


Real-time and historical performance analysis é 


Calendar views of Top SQL, blocks and deadlocks 


Disk activity, latency, and capacity monitoring * One-click and automated tracing with Quick Trace™ 


Free Trial Download: sqlsentry.net/sql-performance SENTRY 
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SQL diagnostic manager monitors your mirrored SOL Server environment: 
* Monitor performance of mirrored databases 


* Receive alerts when the state of your mirrored environment changes 


e Diagnose causes of performance problems in your mirrored environment 
* Take corrective or administrative actions 


... All from a single console! 


With over 100,000 SQL Servers monitored worldwide, Idera leads the market in SQL Server performance and diagnostics solutions 


Our products are easy to use and low-impact. What's more, you'll be up and running in minutes! Www.idera.com 


